Monday, 23 March 2015

RIP, Mr. Lee Kuan Yew

I finished reading the fantastic book Trespassing on Einstein's Lawn by Amanda Gefter yesterday. In the book, she presents the arguments that lead to the conclusion that in physics, there is nothing that is invariant in all reference frames. In other words, there is no ultimate reality; all reality is observer-dependent. This point of view particularly resonates with me because I believe that every person has a perspective on things that to him or her seems correct. Of course, in our cultural and social norms, we do have a pseudo-ultimate reality enshrined in our traditions, laws and etiquette, but these are man-made constraints and not inviolable otherwise.

What is the point of this? The point is that while many eulogies and obituaries will be written for Mr. Lee Kuan Yew, Singapore's first prime minister who passed away this morning, my perspective of living in Singapore while not being born and brought up here helps me understand the enormity of what he managed to accomplish. My point of view of what Singapore is today is influenced by the fact that having grown up in Mumbai, I can see the different directions the story could have gone if not for this man of iron will.

Let's start with the similarities. Mumbai and Singapore are both islands of roughly the same area (600 vs 700 sq. km.). Both were ruled by the British before independence, both have the advantage of being natural ports that helped foster initial settlements and trade. Both attained independence within two decades of each other. Emerging from the convoluted process of gaining independence, both were tried by communal tensions (Mumbai indirectly due to Partition, Singapore between the Malay and Chinese communities).

Of course, this isn't to say that Mumbai and Singapore are identical. Mumbai is a part of a much larger country and policies that influence the nation may not have a directly beneficial effect on the city. Combine this with the tri-level governance of nation, state and city and the sheer force of population makes it a wondrous story in its own right.

On the other hand, the lack of being part of a larger country was also a big handicap for Singapore. With no natural resources, no agriculture, no oil, no minerals, no drinking water, there was virtually no hope for this tiny city-state to survive. The only three things the new nation possessed were its port, its existing British infrastructure in the downtown area and its people - a largely uneducated and illiterate populace of immigrants.

As a commenter on Reddit said, given these initial conditions, I would have restarted my game of Civ4. Enter one of the greatest gamers ever seen - Lee Kuan Yew. After crying on television about Singapore being thrown out of the Malaysian Federation, he set about the task of building Singapore up. He was a visionary like no other.

While most emerging nations are hesitant about foreign investment fearing foreign influence, he encouraged it because he saw that Singapore had nothing it could export or manufacture on its own. Backed into a corner, he encouraged foreign companies to set up base in Singapore, offering them tax incentives and ease of doing business. The experience gained by the population from working in these companies helped local banks and technology companies climb to be shoulder-to-shoulder with some of the best in the world. Think about that: the man saw that the only option was to take which most other countries in its place feared, and he took it and played it so that it benefited the country.

Faced with a severe housing shortage in the 50s and 60s, he set up the Housing Development Board (HDB) to make sure that most people could own their houses and feel they own a part of the nation. It is thanks to his vision that fifty years later, Singapore ranks as one of the highest in the world in terms of home ownership rate (>90%). While the technicality of the term "home ownership" may be debated - all HDB flats are technically on a 99-year lease from the Board - it is nevertheless a long-term security that citizens possess. Contrast this with MHADA, where the mere name conjures up pictures of run-down, cramped buildings.

Faced with a lack of drinking water and importing water from Malaysia, he invested in creating water catchment areas on the island, recycling water as well as desalination technologies (all of which were years away from being commercially viable) as a result of which Singapore aims to be completely self-sufficient water-wise by 2061. Think about that: An island with barely any drinking water and limited land aims to have enough water to support a population of millions. Facing a local populace not tied together by language, he made education in English compulsory (along with a mother tongue language) as that is the language of science, technology and business worldwide. Contrast that with the language politics that we see in India with English and Western culture looked upon as destroying all that is good and holy. To attract top talent to government and keep it corruption-free, he pegged cabinet members' salaries to that of top-earning executives in private companies. Look at the beauty of that idea: if you can earn that much, there is less need for you to be corrupt when you wield power and earning that much can be an incentive for you to get into government, not just the need to do good for the people.

I could go on with other examples of vision and foresight that LKY had, but there was also the other less positive side of things. He ruled with an iron fist, bankrupting opponents (thereby making them ineligible to run for office) via costly lawsuits, controlling the media and imprisoning whoever was considered to be an internal threat to Singapore. We still see the lingering effects of these even though some of the policies have loosened thanks to the democratization of speech via the Internet. Nevertheless, the way I look at it, his view was that the ends justify the means. We try something and if it works, fine; if not, throw it away, try something else.

Yes, in terms of conventionally defined Western freedoms, Singapore lags far behind. This however begs the question: are such freedoms good for their own sake? I will leave the philosophers and the policy-pundits to debate this, but I believe that from the survivalist tendencies that Singapore emerged, this was the best way forward and we are gradually seeing a change in the idea of freedoms here. Freedom of expression and speech engender creativity and the incipient arts and culture scene and the increasingly vocal political space are examples of areas where boundaries are being pushed and modified. It will be a test of the current generation of leaders if they can maintain the trajectory that Singapore was launched on, while still adapting to the times.

As more or less an internal observer with an external background, I see LKY's policies' influence wherever I go. Whether it is taking a flight out of Changi (he pushed for an airport at the edge of the island, when multiple studies advocated the expansion of an existing inland airport like Sahar), taking the MRT to work (the largest public works project of its time, pushed by LKY because he believed that land was too precious for an all-bus system), heading into the research cluster of Singapore (born out of the Industrial Research Unit under the Economic Development Board set up by him), having lunch at a food court or hawker centre (LKY's government organized the food services sectors by setting up markets and formal hawker centres in housing and commercial estates) or going for a walk along the park connector (LKY pushed for greening Singapore to such a degree, that the motto for the National Parks Board today is not Garden City, but City in a Garden), I see the gigantic influence he has had on the place I live. More importantly, I am also able to see how easily things could have gone south. Giving in to special interests, pandering to communities on the basis of race or religion, copying principles from other countries without pausing to consider local applicability or rejecting viable ideas because they didn't emerge internally would all have sabotaged one or more of these.

And that prescient understanding of Singapore and its position in the world along with an iron will while being open to trying new things is what made him such an effective leader. Singapore is truly poorer for having lost its greatest caretaker and his loss will resonate for years as people stop to ask, "What would LKY have done?"

RIP, Lee Kuan Yew.

Wednesday, 4 February 2015

Headspace

I got my hands on a Bluetooth headset (the very comfortable LG HBS-750) a few weeks ago for my commute. I always prefer wireless headsets in the train as there are no wires to be pulled or tangled in the crush. Another reason I enjoy my personal audio in the train is that it gives me much needed headspace, especially when decompressing from work.

I have started listening to some podcasts so I'll chronicle them below. Any recommendations would definitely be welcome. As an aside, I use Player FM as my podcast app, and it is brilliant with a well-designed UI, an extensive catalogue and good offline storage settings.

So, here are some of my favorite podcasts currently:

1. Serial: Well, of course! How can I not be listening to Serial given the hype surrounding it? For those who don't know, it is a long-form weekly podcast spanning almost three months that investigates the alleged murder of Hae Min Lee by her ex-boyfriend Adnan Syed in 1999. The podcast takes us through mountains of unreliable data and dozens of twists and turns with no clear end in sight. It is probably what a lot of criminal investigations are like, and to witness the uncertainty and inconsistencies in the case presented by either side is very edifying.

2. TwiT: This was an unusual one for me. I didn't really get into it at the beginning, because it seemed too slow for what I considered to be a tech news program. After giving it some time however, I realized it is more of a group discussion and has tons of banter that really liven up a two-hour long show. It has become one of the must-hear podcasts for me, and I look forward to it every Tuesday (Singapore time).

3. The Bugle Podcast: From the titans of comedy John Oliver (of Last Week Tonight fame) and Andy Zaltzman (of Cricinfo fame), this self-described "audio newspaper for a visual world" is irreverent and holds no punches in taking everything and everyone to task. I am guaranteed a good chuckle during my commute, much to the disconcert of my co-passengers.

4. No Such Thing As A Fish: Another British podcast, this time from the writers of the comedy panel show QI, it covers interesting facts every week. Loads of comedy and you do get to learn some interesting stuff. For instance, did you know that after the moon landing, Buzz Aldrin worked as a car salesman in a Cadillac dealership and didn't sell a single car in six months? This, of course, led the panel to question whether it was a good decision to have the dealership on the moon, but that's a different story. The name of the podcast comes from the fact that the common ancestor of what we call fish is also the common ancestor of all four-legged, land vertebrates and therefore, there is really no such thing as a fish.

So, that's all from me for now. Do you have any podcasts that you'd recommend for me to add on? I'm still on the lookout for a daily news program that is available between 4AM and 7AM SGT, so that I can get some current news on my way to work. Something like BBC Asia would work well but I can't seem to find a good, relevant podcast. All suggestions are welcome!

Friday, 21 November 2014

A Picture is Worth A Thousand Words (Literally)

I have a healthy skepticism of the part of the job interview process where applicants are asked to write a super-efficient piece of code to create what amounts to a cog in a much larger machine. In my experience as a researcher and developer, most of my time is spent finding the cogs bringing the machine to a grinding halt, also known as debugging, also known as staring at sections of code for minutes on end as your monitor goes to sleep.

One of my favorite methods of debugging is to visualize what I am dealing with wherever possible. Indeed, one of the reasons I got into the field of image processing and computer vision was because generally, when you don't know what's going on, you can look at an image output and get a pretty good idea of what's happening.

Recently, I have been looking into evaluating a classification technique that I have been working with on a text categorization problem. I am familiar with the fundamentals of text classification, as visual classification has long been inspired by the former, but have never worked in the domain before. In order to make what follows a bit easier for non-CS people to follow, I'll try to explain the classification problem in simpler, qualitative terms. Feel free to skim through it if you're comfortable with the field.

Who's who?

So here I am looking at a collection of about 5,000 webpages crawled from the websites of multiple universities. The collection is split into a training set and a testing set. As the names imply, the former is used for teaching our classification algorithm what it is to recognize and the latter is used for testing its performance. The actual categorization problem is a bit more complex, but for the purpose of explanation I'll say we want to distinguish between webpages belonging to faculty (the positive class) from the webpages belonging to non-faculty (the negative class, comprising of students, projects, clubs, etc.). Each document is labelled with its class, and the labels are supplied to the classifier for training, but only used for comparing its outputs in testing.

How do we go about this? We need to structure each document into a form from which its meaning can be easily extracted. One of the most popular techniques from years of research in text analysis is to simply take all the words appearing in a document and count their occurrence. The order of the words is ignored, and the output we get is basically a count of how many times a word in an arbitrary vocabulary or dictionary appears in the document. This is termed as a document vector. There may be additional preprocessing and normalization steps involved, but I'll skip those for now. As an example, a document like:
apple banana apple papaya
will result in a vector:
apple=2 banana=1 orange=0 papaya=1 watermelon=0 ...
for a fruit-based dictionary. Each vector thus supplies the classifier with information about which words occur in the document and how frequently. The classifier then looks for common words in documents belonging to the same class, and uses them to try to assign new, unseen documents to the correct classes.

So what's this got to do with visualization?

Now, the problem I was running into was that my algorithm kept giving me excellent performance on the training set, but not on the testing set. This is very bad for any classifier because it means that it is not able to work on anything it hasn't seen before, making it useless. It is not an uncommon issue though. The classifier is ending up overfitting the data; it is placing too much emphasis on the data it has been given at the expense of leaving little wiggle room to deal with ambiguity in unseen data. This is like the anecdote of the kid who complains to his mother that he could not solve an addition problem in class because he had been taught to do it with apples and the problem asked him to add oranges. Similarly, what we want the classifier to realize is that it is certain words that matter, not all of them.

So, I go about trying to fix the apparent overfitting. This is done by something called regularization, which is a fancy word for penalizing your algorithm when it tries to improve its performance by reducing its flexibility too much. I tried different values for the regularization parameters, different regularization techniques, different parameters for other components of my algorithm. Nothing worked.

Puzzled but quite sure that something should have worked by now, I thought of checking the data. But 5,000 documents containing some 7,500 words overall are not easy to wade through. That's when my instincts kicked in, and I went to the comforting familiarity of images.

Visualization of the positive training set

I visualized the entire positive class of document vectors in the training set as the image above. The vertical dimension consists of the words in the vocabulary, the horizontal dimension consists of the documents, and the brightness of each pixel tells us how frequently a particular word occurs in the corresponding document. There is some normalization done as well, which is responsible for the relatively uniform brightness you see in the horizontal dimension.

Notice there are some bright rows in the picture. I looked back into the vocabulary for the words these rows correspond to and these turned up being words such as "professor", "faculty", "department", "research", etc. - dead giveaways for the positive class, especially when more than one of them occurs on the webpage.

Now, let's take a look at the testing set of the positive class.

Visualization of the positive testing set
Notice the lack of bright horizontal lines or any significant structure for that matter. This immediately violates one of the preconditions for successful classification - the distribution of words in the training and testing sets should be approximately the same (the other condition is that there should be some distinction between the positive and negative classes). No classifier can do well on such a problem, because there is no information about the testing set that can be learned from the training set.

That long-winded explanation is how I realized that there was something wrong with the data. I checked and yup, there was some corrupt data that had caused the document vectors to be improperly calculated. Once that was fixed, the correlation between training and testing performance immediately manifested itself.

Moral of the story? Don't trust the data, especially if you're in unfamiliar territory.

Monday, 10 November 2014

Greetings, planet!


What? Did you really think I wasn't going to open with some variation on "Hello, world!"?

Anyway, I just thought of starting a blog, because I certainly don't tweet or share as much as I used to. Ironic, considering that I am working on social media analysis these days. Moreover, I read this wonderful article on Lifehacker the other day, which says that no matter how little you think you know about something, you are still an expert to someone. Therefore, I decided to start getting some of my thoughts out there, and hopefully generate some new ideas.

I really don't know what this blog is going to be about. Maybe some personal stuff, maybe some Formula 1, maybe some computer vision stuff. Of course, without any idea of what I want this to be, but not wanting to add another blog to the catch-all of "Random Thoughts", I decided to go a bit technical with the title.

I'll try to post regularly and see how I take to blogging, so...