Siddharth Kannan's Blog

OpenCode is Remarkable

2026-03-11T00:00:00+00:00

I started using OpenCode at work recently. It is a very usable tool and its capabilities are remarkable! Its biggest feature is something that can be understood only by those who experience it once: It has the best user experience out of every single AI tool that I have used before. It is easy to understand conceptually as well. It starts as a Terminal UI program. It runs completely locally¹ and it has access to all the files on your filesystem². It is able to run commands using the shell: So, it can run git, grep, rspec, go test, etc. This appears basic but it is a huge improvement over all existing AI tools, because one of the questions I am always asking myself when using tools like Claude or Gemini is “What is part of your context?”, “What do you have access to?” If I ask “What was the final conclusion of <ticket link>?” and the machine responds with “Sorry, I don’t have access to <ticket link>. Can you paste the contents as text here?”, that is a problem. Honestly, I cannot think of a machine generated response that is more infuriating than this. This basic lack of user clarity about context is why I had very little confidence that LLM-based tools had any real capability beyond being used in chat bots preloaded with information from 2021. Until now. Using LLMs through a terminal UI is every computer worker’s best option for improving their productivity at work today.

Another reason I like OpenCode is that its values align with many of my own regarding what constitutes good software. OpenCode is:

Open-source
Capable of running completely locally, without relying on the Internet
Being developed in the open
Not backed by a company whose incentives are to increase token usage
Inter-operable with many model providers

OpenCode stores everything related to its internal state in a Sqlite database locally. This makes it easy to start and resume sessions. (If I had to start OpenCode in a container, the only directories that would have to be mounted would be the one containing this database and OpenCode’s configuration which is a JSON file.)

Adherence to these values have undoubtedly contributed to how good the end product is. OpenCode is supposed to be similar to Claude Code³ (I am not sure which one came first.) Now, I understand why my friends were talking so much about Claude Code as early as August 2025.

The list of built-in tools in OpenCode is limited: 15 tools such as bash, grep, webfetch. However, the single tool bash allows it to do a lot of things: Given the right credentials, it can use the CLI tool of your favorite website and read things from that website. This works well for GitHub and GitLab, both of which have APIs that allow bots to do most anything that a user can. This is the cleanest sandbox that I can think of: If I start OpenCode with a read-only API token for GitLab, I know with certainty that no matter how bezerk the LLM goes, I am protected from catastrophic write or delete operations. There is also the ability to set permissions that force the program to ask for the user’s approval before using a tool, or to even completely disable a tool such as webfetch, forcing OpenCode to be completely local.

Yet another good thing is that OpenCode is designed as a client-and-server program: When you run the CLI command opencode locally, it starts a server in the background, and a client in the foreground. The client that I am working on is the Terminal UI and I like it a lot. However, this background server can have other clients. Clients already exist for editors like VSCode. I am looking forward to an Emacs client; probably built by someone in the community. Emacs can render complex UIs, so a simple chat bot UI should work well. This would be a key contribution to making Emacs the most versatile kitchen sink ever created!

No tool is perfect. The corollary of that axiom is that even OpenCode leaves some things to be desired. The main thing that has been missing for me is a proper sandbox: A great sandbox will reliably restrict the tool from doing what I definitely don’t want it do:

I don’t want it to use sudo on my machine
When I am using it in Plan mode, I don’t want it to write anything at all
When I am using it in Build mode, I don’t want it to write anything outside of the directory where I started it

These are features that are built into OpenCode; but they are not guarantees. For me, a guarantee works at a lower layer. For instance: If I mount a directory as a read-only volume into a Docker container, I know that no program which runs in the Docker container can edit files in that directory. This increases my confidence and makes it easier to experiment. OpenCode’s Build mode asks for permission when attempting to run commands outside the directory where it was started, but it has access to all files on the filesystem already. So, the request-approval sequence is more of a formality and something that an LLM could sidestep. I don’t know when the LLM may decide to sidestep this sequence, but it is possible and that’s what makes me uncomfortable.

An inherent risk of using non-deterministic software (like LLMs) is that there is no telling what the model might decide to do: What if it attempts to use its Bash tool to sudo rm -rf /? I don’t know whether or why it would do that. But it is possible and I do not want to take that risk. So far, this has prevented me from using this amazing tool on my personal machine.⁴ There are some projects out there which put OpenCode in a sandbox: fluxbase-eu/opencode-docker and ocvm. I have not yet tested any of these. Once I find a solution which introduces a sandbox, while still maintaining the great UX, then I would be looking to build the following two wrappers around OpenCode:

$ opencode_plan

$ opencode_build

These should work as “true” wrappers, so that I can start a session in the Plan mode, but later, switch to the Build mode using opencode_build -s <session-ID>.

—

OpenCode has improved my productivity at work. It has made it possible to complete tasks which had been on the “back burner” for months. It has made it trivial to submit improvements which would have taken hours if completed manually. It has significantly improved my velocity in terms of generating code and submitting concise change requests. I look forward to using this tool to review change requests as well, and post an update here in a few months.

You can connect it to any model provider of your choice or run models locally too. I have been running it with the model provider that is offered at work. ↩
Please hold all your concerns about Security. I have them too and that’s why I have not run this tool yet on my local machine and I won’t run it until I have a convincing containerized setup where I can mount local files as a read-only volume. ↩
The only FAQ on the OpenCode GitHub repository is “How is this different from Claude Code? => It’s very similar to Claude Code in terms of capability.” ↩
I have been feeling the urge to use it for some tasks that have been on my list for several months now which have seen absolutely no progress because most of them are tedious and time-consuming, but not particularly difficult. ↩

(What) Is This An Ad (For)?

2025-11-29T00:00:00+00:00

Everything on the internet is an ad for something today. Short videos send people where they can buy clothes, cosmetics, accessories, or install mobile apps. Longer videos try to get people to buy outdated technology. Now, that this is the norm, it may be difficult to remember a time when content was not like this. Casey Neistat’s work on YouTube is a prime example: He started a vlog to sell the idea of his video sharing app Beme. It was a video sharing app, of which there were many already. So, he had to attach a different story and sell it in a different way. The first 100 or so videos of his vlog were an earnest attempt to get viewership, which could later be translated into Beme’s userbase. That plan didn’t quite work; Beme was eventually acquired by some big media company. Was that series of videos any different from all his recent videos where he is invariably trying to sell something?

I don’t know. I can’t really find a convincing argument that it was different. Yes, it was a longer ploy. Producing hundreds of videos in order to market some product in the future is not something one takes on lightly. The cynical take would be that it was always just that, a trick to manipulate users into forming a connection with the person first, and then selling the product. This is not unheard of, I guess it is standard sales practice? I am not that cynical. Perhaps it was a novel method of fusing art and commerce, a method which had not existed before because no compatible medium existed, but not any more cynical than other forms of advertising.

The glaring commonality among the earlier videos and the recent advertisements is the style: the writing using markers on large white canvases, the small color paper cutouts, graphs which represent various things, footage which shows all of this process, the heat gun, the spray paint, the ladder, the meticulously labeled plastic boxes full of supplies, and his studio. All of this has remained unchanged.

Take a form that was once associated with wholesome, non-commercial content, and paint it with the dull, yellowish hue of commerce. This will certainly irk viewers. The form remains distinctly recognizable, but it has lost its luster and its uniqueness.

I have not followed any other creator as closely, or for as long, as I followed Casey Neistat’s YouTube channel. So, I can’t say whether other content creators have gone through this shift. In the case of H3H3 Productions, form and content drastically changed around the same time. In the case of Johnny Harris, content changed and form evolved into something unrecognizable.

Given enough time, all content devolves into advertisement.

Post 10 - Inflation-adjusted Rate of Return

2025-10-04T00:00:00+00:00

Once all your finances are tracked digitally, the logical next step is to ask questions of the data. If you invest regularly, the most common question would be: What is the return on those investments? There are two kinds of returns: nominal return and real return. Nominal return is easy to calculate: ((Current value) - (Original value)) / (Original value). Converting this number into a fictional “annual” rate (CAGR) and comparing it against other assets that promise fixed annual returns (such as time deposits) is useful too. The other type of return that can be calculated is the real return, or the inflation-adjusted rate of return. This gives a better picture over a longer time horizon, when the underlying cost of things is likely to change significantly. In my opinion, one should not obsess over the precise numbers (unless they are very bad, in which case you absolutely should obsess over the numbers). Rather, focus on the process: make it easy to bring data in, take data out, and analyze the data. This post is about how I found inflation data for India, which I intend to use to calculate inflation-adjusted returns.

Disclaimer: This post is not financial or investment advice.

Inflation simplifies everything that is happening in an economy, i.e. the economic actions of millions of people, into a single number. The year-on-year inflation in India in August 2025 was 2.46%.¹ Does this mean that everything is 2.46% more expensive? No. It means that the basket of items that make up the average household in all of India got 2.46% more expensive compared to last year. The things in your basket are probably completely different from the things in the average basket. The concept is abstract, so the numbers are not directly applicable in any sense. But they do indicate that things are getting more expensive in a very general sort of way.

In India, Inflation is calculated using the Consumer Price Index (CPI). CPI is a weighted average. CPI represents the cost of buying a basket containing all the items that a household will buy during a year, against a reference year. CPI is an “index”, so, it has no units. In India, the reference year is 2012 and the CPI value for 2012 is set at 100. So, whatever the cost of the basket was in 2012 was set to 100 and this is what the cost will be compared to in future years. In January 2013, CPI was 104 in Urban India: So, the basket was 4% more expensive compared to the previous year.² … and so on.

Weighted averages seem to be the only statistical tool being used here. The process looks like it has two major steps:

First, calculate the cost of the basket in the reference year:

Consumer Expenditure Survey (CES): Survey many households across rural and urban parts of each state and find out how much they are spending on a variety of items
Item selection: Select a set of items from the CES which will make up the CPI basket
Weight Reference Year: Use data from the CES to calculate the weight of each of the selected items in the CPI basket
1. Repeat for the rural and urban parts of each state
Price Reference Year: In each state, select many villages. Select markets in these villages. Find the price of all items that were selected in (2). Use a mean of these prices as the price of an item in rural parts of the state.
1. Repeat for rural and urban parts of each state
Calculate the base basket cost: Using the list of items selected (2), the price of each item in (2), and the weight surveyed in (3), calculate the base price of the basket.
1. Repeat for rural and urban parts of each state
2. This is the value which will be indexed to 100

Combined CPI is a weighted average of the Rural and Urban CPI, with different weights for each state, depending on how much of the spending comes from rural and urban areas.³

Have you had a sufficient amount of weighted average math and summation formulas?

(I have not gone into the calculation of the headline number, which is usually reported in the news: “All India” CPI for rural, urban, and combined. This is because the process seems to be different for rural/urban and combined, and I need more time to look into this.⁴

The weights⁵ were set in 2015 based on a 2011-12 survey of an urban and rural household’s monthly average expenditure.⁶ The weights suggest some patterns, but they have to be taken for granted as “inputs” to the model. For instance, the sub-group “Fruits” accounts for 10% of Urban CPI and 6% of the Rural CPI.⁷ These weights are unlikely to match any actual household’s consumption patterns, because they are geometric means of all the households surveyed.

There is a bunch more complexity in the calculation of the final CPI. I have questions such as:

Where does the price of a fruit come from? What if the fruit is not in season?
If it is a shop, is it the same shop each time? What if the shop closes?
What if the item is replaced with another item? For instance, 6.1.06.1.1.02.X: toothpaste, toothbrush, comb, etc. seems like a composite item which includes every brand of toothpaste and toothbrush. How?
There are 300 items⁸ in the table and the cost of each of these items is required to calculate CPI. Who goes and finds the price of all the items?
… so on.

Almost all of these questions are answered in CPI: Changes in the Revised Series but that is 80 pages long and I have only gotten about 25% of the way through it. So, I will look forward to reading the rest later. For now, pushing all of those questions to the back of my mind, I can still convince myself that I should rely on the All India CPI (Combined) (i.e. the headline inflation number) for these reasons:

It is the only way that I know of to simplify a complex system which is beyond comprehension
Something is better than nothing!

Now that we know how we arrive at the All India CPI (Combined)⁹, let’s get to using this number:

CPI is released monthly. It denotes the price of one basket of goods during the previous month.

Number of baskets I can buy today

(Current basket count) = (Current investment value) / (Latest month CPI)

Number of baskets I could have bought before investing

(Original basket count) = (Original investment value) / (CPI for the month of investment)

Real return

= (Current basket count) / (Original basket count) - 1

The real return is a percentage. If your investment strategy has managed to beat inflation, then the real return will be positive. If not, it will be 0 or negative. If we have the monthly value of our investments, then it is possible to calculate the real return on a monthly basis and watch the trend: This would be particularly useful: If the current value is negative, and it has stayed below 0 for the past 4 years, you should do something. If the current value is negative but it was positive for most of the previous year, you are probably OK. (Not investment advice!)

This works for a single investment, which has not been redeemed in part, or increased, during the intervening time. Investment patterns rarely work like that: We would need to use the Internal Rate of Return (IRR). IRR is another confusing concept; I don’t fully understand the math behind it yet, so I have shied away from using it. With irregular cashflows in and out, IRR is the recommended metric though. Nominal IRR is easy to calculate using a tool such as Ledger or a spreadsheet program. Using CPI, I could reduce everything to the base year’s prices (2012 prices) and then, use IRR to calculate the Real IRR.

Source: Press Release:Press Information Bureau ↩
Source: Current CPI series with base 2012 = 100. ↩
Table 3 in https://cpi.mospi.gov.in/PDFile/CPI-Changes_in_the_Revised_Series.pdf ↩
See Annexure-III and Annexure-IV. Please e-mail me know if you understand it all! https://cpi.mospi.gov.in/PDFile/CPI-Changes_in_the_Revised_Series.pdf ↩
Source: All India Item Combined Weight Base:2012 ↩
“Weighing diagram gives the share of each item in the total consumption expenditure in a CES. The weighing diagrams for the CPI series (Base 2012=100) have been derived on the basis of average monthly consumer expenditure of an urban/rural household obtained from MMRP data of NSS 68th round Consumer Expenditure Survey (2011-12).” Source: https://cpi.mospi.gov.in/PDFile/CPI-Changes_in_the_Revised_Series.pdf ↩
Table 6 in https://cpi.mospi.gov.in/PDFile/CPI-Changes_in_the_Revised_Series.pdf ↩
Annexure III in https://cpi.mospi.gov.in/PDFile/CPI-Changes_in_the_Revised_Series.pdf ↩
I am joking: We have skipped most of the useful parts. ↩

Post 9 - Metabase and Sankey Diagrams

2025-08-14T00:00:00+00:00

In the past week, I tried Metabase, an open-source data analysis tool, which can be started as a single Docker container and used in the browser. My final goal is to turn it into a setup where I can upload the CSV export of my GnuCash file and have preconfigured dashboards show me data that I am interested in; maybe even a Sankey chart of my cash flow report, so I don’t have to manually go through the numbers. I have talked about GnuCash twice on this blog before. Both of those were steps leading up to this attempt, where I am trying to plug GnuCash into a “real” analytics tool, rather than making do with GnuCash’s in-built reports functionality or the solution with converting GnuCash into Ledger, and using Ledger’s CSV output feature and pasting the output into a spreadsheet. I am certain that there is nothing out there which does this already. (If you know of something, please e-mail me!) I have no idea how long this will take or whether it will even work. As always, it has already proven to be an interesting exploration.

Metabase is an open-source data analytics application. It uses the browser as its “interface” and it is one of the new-age products which attempts to make dashboarding easier for business metrics. A product in this family that I have a passing acquantaince with is Looker, and its cousin, Looker Studio. I built dashboards using Looker Studio which would run a query on BigQuery on anonymized user data that was imported from MySQL, which was the main data store for the backend application. I have been hearing about Metabase lately¹ and I wanted to test it out. It has two features that I like a lot:

It can be started up as a Docker container running locally
It is possible to upload CSV files as data (rather than having a “live” database connection to MySQL or Postgres)

Both of these are requirements for me because I don’t want to upload a CSV export to online platforms like Looker Studio or BigQuery, and because I don’t want to go through the extra step of exporting GnuCash to CSV first, and then importing that CSV into a live database. The second step is something I can live with, but I will probably stop using any setup which requires too much work. (I posted about a duct-taped setup with several tools to convert online articles into ePub files about two years ago. That setup lasted about 2 months before I gave up on it completely, because there were too many manual steps before I could connect my Kindle and sync the Calibre library with it.)

I tested out Metabase with a sample GnuCash book that I created with completely made up transactions and it works very well. The product is very polished; it can be started easily using a simple Docker Compose file, which is there in the documentation. It requires a Postgres container, which is where all the state is stored. For the kind of analysis that I am looking to do, I am going to use the same Postgres container for the Metabase application DB (i.e. Metabase’ state) and the data that I want to analyze.

Importing CSV files worked after a few changes to it: The CSV file needs to be encoded in UTF-8, which is reasonable. Once uploaded, columns are automatically detected, and a “Model” is created (which is the equivalent of a Postgres table for our rudimentary use-case). It becomes possible to make some simple “Questions”, which are automatically updating calculations based on a Model. As it is possible to replace all the data in a Model with a new CSV file, the Questions will also update and show the result of applying some aggregations on the new data. Neat!

I imported a file with dates in it, but I did not see Metabase treat those dates as a special type (by default). This might be something that I would have to look into, if I start thinking seriously about building a setup. The Metabase documentation does mention that being able to filter by time periods is a feature. I wonder whether the problem was that the dates in my input file were in the YYYY/MM/DD format rather than the ISO YYYY-MM-DD format.

All-in-all, I was very happy with this. I toyed with the tool for about 2 hours, and this was enough for me to get a hang of the basics, and be able to make some easy aggregations. Exploring the documentation though, I hit on gold: Metabase can create Sankey charts from CSV data.

Sankey charts are a great way to display pie-charts, where one can “drill down” several times into each chunk in the pie, without an interactive interface. This is what they look:

Source: Data from 2021 – Understanding the current energy situation in Japan (Part 1). Built using SankeyMATIC: Build a Sankey Diagram. You can edit the input data for this diagram here.

These kind of Sankey charts exist for financial data as well. Most commonly, they exist for public companies, such as this one. This simplifies a (possibly) huge spreadsheet into a single diagram which is easy to understand at first glance; no explanation is required. It shows both sides of the “sales” diagram: the products that sales come from, as well as what happens to the proceeds. Sankey diagrams are great!

I have wanted a similar visualization for my (rather simple) GnuCash data as well. GnuCash is a great piece of software; however, its Reports functionality is pretty feature incomplete. The in-built reports are fairly limited in type. Even those that exist don’t have enough flexibility to make a report look the way that one would want it to: some reports have the ability to convert raw numbers into percentages, while others (like the Cash Flow report) don’t. It is very hit-and-miss, and to the extent that I do any “analysis” of my GnuCash data I do it either after converting the file to Ledger, or by manually copying a table from a report into a spreadsheet and working with it there. There is no way to even export reports as CSV files, so that they may be plugged into another data analytics platform. Ultimately, this all adds to the “manual” nature of the whole task; causing me to do it less often than I would actually like to.

I’m hoping that Metabase (or Duck DB, which is one alternative which I have not looked into yet) will help me do analytics directly on the CSV export, or better still, the GnuCash file itself², and that one (or both of them together) will help me visualize Cash Flow as a Sankey diagram.

Mainly, thanks to Minusx - Metabase AI. ↩
The GnuCash file is a gzipped XML file. So, gzip -d will give you the plain XML file. ↩

Post 8 - Civilization 6 (Game)

2025-08-10T00:00:00+00:00

I have been spending a few minutes every week playing the Civilization 6 game. It is a lot of fun. I am about 300 turns into a 500 turn game; the shape and size of the game world is clear to me, the major civilizations have taken shape, and now, it appears to be a race to getting the right technologies, and units, to defend territory and also, “make” it seem well-defended to the game. The game engine catches on very quickly if it notices that some part of the territory is not well-defended and spawns Barbarians right next to it. Moving troops around has also become much easier at this stage, due to the advent of modern roads and units with more movement. If you have not heard about Civilization before, then none of that would make any sense to you, and I highly recommend this game to you.

I started playing Civilization to find out more about what “strategy games” really were all about. I have an interest in history, and my thought was that these strategy games would give me a clearer idea about how planning works when there are multiple cities and 10s or 100s of people involved. That has been the case, to some extent. There are some constraints in the game which can simply not be overcome: The movement of a unit is limited, the number of turns (amount of time) that it will take to move a set of units from one part of the map to another is fixed and nothing can be done to really bring about a “sudden assault” at any time that one may wish. I am sure that I am making huge mistakes in terms of strategy; mainly, being too impulsive and eager, rather than calculating and patient.

The times that I have been patient, the game rewards the player generously: When attempting to take over a City-State in the game, for instance, a well-planned strategy with unit movements for 10-15 turns before actually firing a single round leads to a much quicker end to the take-over rather than a hapazard attempt that starts with a handful of units. I learned this by making mistakes; the game provides a satisfying visual and auditory experience (the soundtrack plays pretty much the whole time and has a hypnotic, looping feel to it: like the instrumental music that plays in a mall, it sounds vaguely familiar but is never repetitive or boring (yet)). I do take issue with the sounds that play whenever I click on a button though: these are intrusive and do not even “fade out” causing a sudden, high pitched sound, like a door slamming somewhere near your ears.

Another moment which was really stunning for me was the first time that I realized that the game world was round! The world is presented as a flat map, and one must slowly explore the map by oneself to identify where other cities and civilizations are. Looking at a flat map with the north and sound having snow and glaciers, I expected the map to be a flat rectangle. In fact, as I kept exploring to the West, I realize that the map wraps around and it was actually a circular world, just like Earth! This was a great moment: I am not sure why I did not expect this simple fact; with a game world that is not round, moving units around would be hugely inefficient and not at all similar to the way that things like trade work in the real world. The shortest route can be taken, and the game calculates this for you depending on where you want to go. Convenient!

I stopped playing for a while in the middle, while I played Last of Us: Part 2¹ and Need for Speed for a few days. (It had been a long time since I had played a car racing game, and I wanted to relive a few of my early PC gaming days! I played the same NFS game, but remastered. When I was looking through gameplay footage on YouTube, I was surprised by how many of the shortcuts I still remembered, despite not having played the original version for at least 10 years now.) I found the middle part of the game to be a bit boring: most of it is just setup for the future, where things will actually start going much faster, with each turn involving a lot of action and decisions. I let my concentration go a bit in this part, so I have probably made some mistakes here. For the city improvements that I built, I did not even realize the great improvements to Production, Housing, and Food provided by districts such as Commercial Hub or Encampment. This will be something that I will be focusing more on in my next playthrough (which might literally be a year from now!)

Another part of the game that I have mostly not explored this time is attempting to get to technologies early by boosting them. I found the constant selection of the technology to research from the Tech tree time consuming and boring at first, but it is the most important part of the game, as the difference in strengths between units of different eras is huge.

But after that small slump, I have picked it back up where I left it off, and it has been satisfying now. I am also spending less time on individual turns, and planning well ahead, by setting up production queues that last 30-40 turns for each city, and selecting items from the Tech and Civics trees for about the same duration too. This leaves only the day-to-day to be handled, and this is usually OK with units that are simply patrolling. One thing that I feel that the game does not do a good job simulating is the real-life existence of deputies, who can take over the management of a small part of the game or a few units. I don’t know if this game has a multiplayer version, but that would be really interesting: Giving over military policy and units to one player, while leaving the decision of trade routes to another player, and so on.

I watched the movie His Girl Friday (1940) and it was very funny! Everyone in this movie talks really fast. This is something which has changed drastically: Today, I would call an actor who is talking slowly as being dramatic, but it looks like that has changed a lot since the beginning of film.

I spoke about the reasons I liked this game in another post. ↩

Post 7 - Using MacOS

2025-08-07T00:00:00+00:00

For the past few years, I had to use a Macbook laptop at my workplace. The corporate policy to only use Apple computers never made much sense to me: The excuse to not support anything except Mac or Windows was the usual “Mobile Device Management requirement” strawman. There are very good MDM solutions for Linux as well now; these solutions, unlike the expensive ones for Mac and Windows, happen to be cross platform. I could not convince anyone of this; nor could I get this policy changed; at some point, I resigned myself to making do with this computer. This post contains a list of applications which I found useful and helped me turn a Macbook into the other Linux machines that I am used to using. If I ever have to use a Mac again, I hope this list will be useful to me then.

rxhanson/Rectangle: This was my pseudo window manager. This app brings the ability to bind keyboard shortcuts to resizing and moving windows around. So, it is possible to bind three sets of keys to the first, middle, and last third of the screen. I used Cmd-Shift- with U, I, and O for this. So, I could use Cmd-Shift-I to snap a window to the middle third of the screen. There is also a way to do this for the first, second, third, and the last 25% of the screen, and the left and right halves. So, by binding the right keys, it is possible to almost get a Tiling window manager experience. The only draw back is that if you have more than 4 windows open, then those windows will be hidden below the ones that are open. I could not really do much about it. I never had more than 4 open: Terminal (Alacritty), Emacs, Slack, browser, were usually the only ones I had open.

Amphetamine: macOS keep-awake utility: A simple utility to keep the screen awake. The MDM was not configured to or failed to prevent this utility from running and keeping the screen awake. (I have heard that there are some requirements for 15 minute screen lock after inactivity, for some certifications. This might also just be an urban myth though, because most of the tools seem to operate with a “security through obscurity” approach, but keeping their workings and how they comply with various requirements a “secret.”

Little Snitch — Network Monitor and Application Firewall: This is another nifty utility that can be used to monitor what connections are being made by the various applications which are running. I used to use a Commercial license for the PHPStorm IDE (because it is impossible to write PHP without an IDE and the LSP for PHP has a long way to go), and the IDE would constantly ping the license server throughout the time that the application was open. There were also many other strange connections which I found using this utility. If I remember correctly, there is some sort of limitation on the free version of this utility, it shuts down after a few hours or something. So, I switched to objective-see/LuLu after some time.

Menu World Time: A small utility that shows the world time for a few different time zones in the menu bar. This one is really nice: If you have meetings or the like, this one can be used as a quick reference. It simply works.

MeetingBar: Another small menu bar utility which will show the current or upcoming calendar event in the MenuBar, and can be used to bind a keyboard shortcut to the action “Join my next meeting.” It supports most of the common video-calling platforms. I used it a bunch. One caveat is that it uses the locally synced Calendar (which is synced using CalDav). This is usually a few minutes behind the actual CalDav source server, even with push notifications enabled. So, if an event changes close to the time at which it was supposed to begin, then this does not work.

What Did Not Work

I also wanted to add a list of some of the utilities which did not work for me.

Karabiner-Elements: An apparently very powerful utility that can be used to remap keybindings freely on Mac OS. I never really figured out how to get it to work. I tried to use it multiple times, with multiple keyboards and setups. But it always seemed like the shortcuts I was setting up were for a single keyboard: that seemed very strange to me. The whole application felt hard to use to me, and I did not really spend the time to figure it out once I realized that Rectangle did a good enough job as it is.

yabai - A tiling window manager for macOS: A tiling window manager for Mac OS. This one has a long way to go. I tried to use it, and it kind of works, but the software needs to be improved a lot before it can really be a daily driver. Nevertheless, I did know at least one engineer who was using this as their daily driver window manager. They used to complain about crashes now and then; but I don’t remember much else. I would say stay away from it.

Dash - API Documentation Browser: I bought a license for Dash around the beginning of my career as a Backend Engineer, because it allowed me to quickly reference functions from the Go standard library. It was a very useful, and simple tool, which aggregated a bunch of documentation (PHP standard library was also included). However, the app changed significantly over time and continued to become heavier and heavier. They also changed their licensing and started putting pop-ups which asked you to buy the license for a more recent version of the application. Over time, I ended up never using it and just relying on the browser and search (or the Go LSP and Emacs integration to jump directly to the standard library and read the comments there).

I fully expect half of the links here to not work by the next time that I switch back to using a Mac OS and having to find alternatives for all the above use-cases again.

Post 6 - Short Term vs. Long Term

2025-08-05T00:00:00+00:00

I hope you have not been following the news. I have not been following it at all, nevertheless, it is very hard to completely shut off the influx of daily happenings from everywhere. One such daily happening is the back-and-forth about a strange thing called “tariffs.” Let’s assume that you know very little about what it is. The only thing that you could take away from the breathless coverage of every flip, backflip, and reverse-backflip, is that tarrifs are bad, wanting to impose them is bad, but also good in some cases, but mostly bad in most cases, except that one case … You get the point: They are a complex subject which can not be covered in a 5 minute article or a 20 second AI-generated summary of that article. This constant influx of information convinces some people that they have to do something, which they use to invest in the stock market. These short-term plays have never meant much for the average retail investor: Bogle’s book expresses that argument eloquently and succinctly.

The TV show Veep captures the illogical nature of this well in a sequence from Season 4:¹ The main character tweets something mistsakenly; her opponent starts criticizing her for this mistake; she decides to blame hackers from another country for this tweet in order to distance herself from the blame; Opposition parties goad her into a real response for this apparent hack (they don’t know the truth!); she imposes sanctions on the other country! The fact that this was all a farce was lost in just one cycle, even on her: She was the one who started the whole thing.

This is precisely how news probably works; someone says something, it gets repeated on TV, everyone grabs onto it and attempts to reduce the standing of the person who said it, so the person who said it responds, and so on … As they employ full-time professionals to market everything that they say, maintain their “image” among the public, and respond to everything that everyone else says, this cycle continues unabated. The news media will report these constant volley of responses with as little context as possible, for as short a time as possible, because people get really bored of watching the same thing over-and-over again. It is amusing to watch this cycle in action. But it is horrifying to watch people buying into the insights and losing money over it.

I have always wondered: Who sells their holdings during the dip? Do the people who sold their holdings really believe that their holdings would never regain their value? Or rather, the technically correct way of putting that would be, (I think) do the sellers believe that the inflation adjusted rate of return on their holdings would be negative due to this single adverse event or announcement?

It seems strange to invest in something that is so fragile: If a single unexpected thing can tank the overall growth of 500 companies forever, then those companies were not very good in the first place. This is obviously not the case. But somehow, the stock market crashes and people always sell during these “mega-announcements.” It happens during elections: Assume that a political party is expected to win the election by pollsters (We’ll come to them later.) Every possible outcome produces an outsized response:

If the party wins in a landslide, market goes up: “No one expected a landslide that big!”
If they win marginally, market goes down: “No one expected the election to be so close”
If they win but the government is formed through a coalition, market goes down: “No one expected a coalition government.”
If they lose, “No one expected them to lose!”

Clearly, if the rate of growth of a company’s profit depends on a single party being in power, maybe don’t invest in the company and drive up its stock in the first place?

I see the problem as being an overall information overload about the short-term, and a perfect information void about the long-term: The importance of events that happened today are elevated to a point where everyone starts believing that they are actually significant. There is a complete lack of information about the long-term, which might act to counter balance the short-term fears and panic.

Let’s see the headlines for a single story from the day that the “big” announcement was made to today, at monthly intervals:²

2025-04-02	Man Unveils Expansive Global Tariffs
2025-05-02	Stock Market on Track to Erases Losses From Man’s ‘Liberation Day’ Tariff Rollout
	Country A Is Considering Trade Talks With Country B, but It Has Conditions
	Why Company C Warned About a $900 Million Tariff Hit
2025-06-02	As Courts Call Tariffs Into Question, Man Again Turns to His Favorite Tool
	Country B’s Dependence on Country A for Rare Earth Magnets Is Causing Shortages
2025-07-02	How Country B - Country C Tariff Talks Got Stuck on Cars and Rice
	Man Says Country B Has Reached Trade Deal With Country D

This is hilarious. All I did was run the search: site:nytimes.com on:2025-04-02 for a few months, pick a few random headlines, and replace all identifiers from the headlines. This is complete chaos. Also, it is completely pointless to follow this on a daily basis: NOTHING HAS HAPPENED YET! This can not be stressed enough. Following this on a daily basis, you might think that something that happened today is important. But if you wait another week, 10 other things would have happened which were actually more important.

I have taken the worst possible example here, one in which the headlines are a result of intentional attention seeking. News people will cover the same subject for a long time only if there are many new twists and turns to “explain” and “break down.” So, like reality TV shows do routinely in their later seasons, when they are no longer able to come up with new ideas for twists and turns, you start introducing arbitrary changes, which are intentionally designed to increase the drama, but in reality, end up making the show boring, tiring, and contrived; acting to take out the essense that people were watching the show for in the first place. That’s what happened with the above story too: Now, analysts have started saying that the market has discounted the effect of anything happening already and these announcements will continue to have less and less of an impact. That is just another way of saying that no matter what announcement comes, no one thinks anything real is going to happen, so they just ignore it and go about their work as usual.

What about the people who sold on the day of the first announcment, 2025-04-02? To put it bluntly, they lost big time.

If the problem is the short-term, what do you do? Is there something else to read instead? No. That’s my opinion; let me know if there is something to read about the long-term. The best I have found are magazines, but even those have started focusing entirely on the short-term. There are textbooks and non-fiction books which explain concepts, but do not connect it back to relatively recent events. There are non-fiction books that examine a past market frenzy, but none is timely enough to read right now. There is fiction which creates arbitrary situations and tries to show the consequences. There are documentaries which explain what happened exactly, but not what is happening since then.

There is no form of media that takes the happenings of the past year, puts it all together, and gives us a balanced prediction for the future. This is something that one must come up with by oneself. Obviously, that’s not possible: No one has all the required ingredients: the resources to get all the data, the expertise required to analyze it, or the time to do it. If you have the expertise, you’re probably doing it as a job anyway and don’t want to spend your free time doing it. If you have the data, then you probably only care about getting the data and not about analyzing it. If you have the time, well, you can get the expertise but you may not have the money because you have time … etc.

There are forecasts from investment banks. I wonder how reliable they are. Putting together past forecasts and comparing them against actual index performance is time consuming, I have tried to do this but basically, these investment bank reports are private. The only way that the public knows about these reports is through reports from financial newspapers. It would not be in the interest of the financial newspapers to compare the investment bank forecasts against reality and show that they are doing poorly, if that is the case. If they are doing well, there’s no point in bragging, because all the investment banks are probably performing at the same level. Right?

So, the only way is to manually comb through articles and find the article that matches. These articles don’t mention the methodology for comparison with real performance either: They just say Index A is forecasted by Bank B to be at Level C in month M of year Y. How could you compare it to the real performance? The 30-day moving average at the end of that month? Maybe. Getting that data for the index you’re tracking in a way that’s easy would be a bit of a task, but nowhere near as difficult as assembling the predictions for an index A from bank B over the past several years at various points during the year. So, these forecasts might be part of the short-term information overload too, existing only to convince the clients of the investment bank to use a strategy that the investment bank is pushing at the moment that the report was written.

It has been a while since I watched the show, so I am not entirely sure which season it is. ↩
I have stripped all identifying information. These headlines are significantly more amusing now. ↩

Post 5 - Emacs and Elisp

2025-08-02T00:00:00+00:00

I started using Emacs around 2020. At the beginning, I really only wanted to use it for its superior support of LSP and Org-mode, which in my opinion, is a far more complete mark-up language compared to Markdown. It is simple to use, but it can be used to do complex things too, if that’s your preference. I am able to have a directory with multiple Org files, TODO items in various files, and all of these can be presented in a familiar daily, weekly, monthly task-list format using org-agenda. I can schedule things, set deadlines, set reminders based on those deadlines, and repeat tasks periodically. These are all things that Org’s design allows: the native integration into Emacs makes it easy to get started using all the functionality provided by Org, without having to depend on external packages. Systems like this probably exist for Markdown too. I never looked for one because my problems with Markdown began well before I found out about Org-mode. Writing Elisp and extending Emacs was something that I started doing because I had no choice. I have not gotten far enough with it yet, but I have been able to implement some useful functions, for which ready-made packages don’t already exist.

unwrap-all remains the most useful of the functions that I put together with fairly rudimentary knowledge of Elisp. The function itself is probably written in a way that is very non-idiomatic Elisp. This function takes a file¹, assuming that it is wrapped at a set text width (say 80 characters) and paragraphs are separated by a newline, it will unwrap all the text so that each paragraph is on a single line, with empty lines separating them. The resulting text works well with most web applications that don’t support Markdown. The tool that was used to fill in self-assessments and peer reviews at work did not support Markdown; people would regularly start writing their peer reviews within the tool and lose it because their browser crashed or the tab refreshed or something. I never considered writing a long block of text within a web browser to be a real option anyway. Without features provided by keyboard shortcuts-based editing, such as being able to jump to the beginning or end of the line, word; or to delete until some character, or the remaining part of the current line, one has to constantly jump between the keyboard and the mouse, which leads to quite a bit of inefficiency. (The tool eventually implemented auto-save to prevent this, but not Markdown support.) After manually doing the wrapping and unwrapping manually for 2 quarters, and then searching online for a local tool or script which could do this², I decided to finally write something that would solve this problem.

The function was surprisingly easy to write. Elisp is esoteric, with terminology such as car and cdr, which refer to [0] and [1:] in the usual Golang notation. In a handful of iterations, I had something that actually worked. As I was writing in Org-mode most of the time, the function handles Org-mode text and does not unwrap things like code-blocks or tables. (Of course, these were not part of my self-assessment. They came in later when I realized that unwrapping was useful even before pasting text into Google Docs. This was before Google Docs had native support for Markdown.

Showing or hiding the word count in the editor’s mode line (bottom bar) was also something that was primarily useful for me during the performance review cycle, when there were guidelines about how long peer reviews generally should be. After having written a bit, being able to see how much I have written and whether I am being too verbose was a useful gauge. Once again, due to Org-mode’s magic, I was able to open a single sub-heading from a much larger file in a separate buffer. This buffer’s modeline would show me only the word count of the text within this sub-heading! This is what I mean when I say that Markdown is not a complete system. This is something that is obvious to me: A single file may have a lot of information but headings separate it, and it should be very easy to treat those headings as distinct files.

One thing that I have started doing recently is to move all of my periodic TODOs into my existing notes directory. These usually include things like logging into a website, and downloading an invoice or a CSV file and storing it somewhere. These actions are often manual and monthly, because there is no API which can be called and the systems that provided these files are well behind the usual trends. Although quite slowly, this system has started to show some value: Lately, I have forgotten about most of the periodic TODO items and I am trusting the Org-Agenda weekly view to show them to me whenever I open that view. This system works well at work, where I am opening my computer on every workday, and checking the agenda. On the personal side of things, since I don’t use my computer every day, or even if I use it, I might not open my agenda, this might need to be tweaked to allow notifications: A systemd unit which runs the org-agenda command and sends a notification using notify-send when there is a TODO which needs to be done today or is overdue, perhaps? These are ideas that I have been exploring.

One of the limitations of Emacs is that it is a single-threaded application. No other modern editor is single-threaded. Editors from JetBrains like RubyMine can run spawn the process which will run tests in the background, bring that output into the editor’s test “visualization” buffer, while keeping the panes that show code browsable. This is not possible with Emacs currently: The unwrap function, for instance, blocks the editor from doing anything until it is complete. This is not a problem with most functions. It is an issue with Org-agenda though: When you run Org-agenda on a directory with 100s of Org files, it takes a lot of time. If you want to measure it, you can do so using Org:

#+begin_src elisp
  (benchmark-elapse
    (org-agenda-list))
#+end_src

When you run C-c C-c with your cursor at end_src above, Org-mode will offer up a prompt to run the Elisp in that block, and if you agree, it will print the results of that Elisp into the same Org file. The result of the benchmark-elapse function is the amount of time that it takes to run the code within that function:

#+RESULTS:
: 146.782507877

This is the number of seconds that org-agenda-list takes when run on a directory with a few 100 Org files. Yes, that’s a lot. And during that whole time, the editor is blocked because (you guessed it!) the main thread is executing this function and is not able to do anything else.

One saving grace is that its only the first execution after Emacs is started which takes so long. Executions after the first one finish in under 10 seconds. I guess there is some caching or tracking of what file is changing, or something. I don’t know why that caching does not work across Emacs sessions; maybe the cache results are stored in the memory of the current Emacs process. 150 seconds (and even 10 seconds for reloading the agenda) is unacceptable.

My guess is that it takes too long for Emacs to figure out which of these 100s of files have TODO items in the first place because it is single threaded. The difference between single-threaded (and battle-tested) Grep and multithreaded ripgrep is well-known. For instance, this is the difference in runtimes:

# Single-threaded
$ time find . -type f | xargs -I{} egrep '(TODO|WAITING|DONE|CANCELED)' {} -l >/dev/null
noglob find . -type f  0.00s user 0.00s system 90% cpu 0.003 total
xargs -I{} egrep '(TODO|WAITING|DONE|CANCELED)' {} -l > /dev/null  1.08s user 0.42s system 103% cpu 1.446 total

# Multi-threaded
$ time rg -w '(TODO|WAITING|DONE|CANCELED)' -l >/dev/null
rg -w '(TODO|WAITING|DONE|CANCELED)' -l > /dev/null  0.04s user 0.01s system 288% cpu 0.017 total

1.446 seconds for grep; 0.017 seconds for ripgrep: roughly 85 times faster.

org-agenda-list uses org-agenda-files as the list of files or directories which it traverses to find all the TODOs. What if I gave it a list that came from rg to start with? Will that speed things up? How much?

I tried it. This oneliner converts the list of files output by Ripgrep into a list for Elisp:

$ rg -w '(TODO|WAITING|DONE|CANCELED)' -l | \
    sed -e 's#^#"~/notes/#g' -e 's#$#"#g' | \
    paste -sd' ' | \
    sed -e "s#^#(setq org-agenda-files '(#g" -e 's#$#))#g'
(setq org-agenda-files '("~/notes/some-file.org" "~/notes/some-other-file.org"))

I added the Elisp equivalent of that oneliner in this commit.

I put this list manually into the Emacs configuration, for testing. And voila: The time taken to run org-agenda-list and present the current week’s agenda drops to about 5 seconds! The time taken to reload the agenda is almost 0, at about 0.28 seconds. The list found by ripgrep contains only 100 files, and each of them contains a TODO item.

As ripgrep itself takes an insignificant amount of time to run, I think I am going to wire up ripgrep to always run and over-write the value of org-agenda-files right before org-agenda is called. I believe that this is possible using advice-add, which allows users to add code that executes before, after, or around existing Elisp functions!

Emacs calls any open file a buffer. Buffers don’t have to be associated with an actual file on disk. ↩
I certainly did not want to put my self-assessment text into some online editor. I am shocked by how many people upload PDFs of personal documents to random websites in order to resize them before sending them to someone else. ↩

Post 4 - Science Fiction and Time

2025-07-30T00:00:00+00:00

Science fiction is a great genre. Within science fiction, the subgenre that manipulates time is my favorite. Movies based on the warping of time, the invention of a time machine, or the strange effects of a single episode of time travel are some of the best. (The other subgenre that I really like is the investigation of fictional accidents, such as the one that is conducted in Airframe, by Crichton) Interestingly, warping time does not need to resort to fiction: the best examples would be Memento, 500 Days of Summer, and the Seinfeld “backwards episode”.¹ These depict the events of a character’s life through an odd, nonlinear narrative. If the linear flow of time is the basis for everything, then it is fascinating to explore the alternate realities and the questions that it raises.

A few science fiction movies that have stayed with me over several years are:

**EVERYTHING BEYOND THIS POINT IS A SPOILER FOR ONE OR MORE OF THE ABOVE. READERS BEWARE!**

Arrival is at the top of my life because it introduced me to a great movie and a great writer (Ted Chiang). I can clearly remember how I ended up watching Arrival in the theater: It was the only English movie playing in a nearby theater during a summer or winter break from college; the trailer gives no hint that the movie itself is based on time: deliberately focusing on the aliens who arrive. I later discovered that the day I watched the movie was also the last show in that theater. The scene in the movie that truly hit me was the one where it is finally revealed that the protagonist has understood the aliens’ language, and is now able to look into the future, just as easily as we are able to remember the past. Even though the movie’s “punchline” was not this revelation but the choice that the protagonist makes, it was this scene where what was going on dawned on me. I have watched the movie over and over several times in the past few years, so the feeling of that fresh dawning is gone for me now. I will perhaps find it again with another movie at some point.

The rub in all these movies is the choice that the creator makes: Will time travel enable the characters to change the past or future, or will it just be a tool that they can use to look into what is always going to happen? In my opinion, the former is easier to sell, but the latter is more appealing to create. Minority Report does it best; the movie begins with complete confidence in the system, but through the chief proponent’s own experience, a flaw is revealed and we are lead to question whether such a system to prevent anything before it happened really is worth it; the world that exists before and after something happens is different enough that the thing being prevented from happening is just as big of a deal as the thing happening. Minority Report still confuses me; was the movie in support of the fictional idea or against it?

An aspect of time travel movies and dramas that keeps me engaged for a very long time is that many of them are utterly inscrutable: No matter how many times you watch them, there is always more to unravel. Primer was a great example of this. I was fairly obsessed with the movie 8 years ago (!!!) I am still unclear about several aspects of the movie, which makes rewatching the movie rewarding experience. Primer was also the first movie which convinced me that a great script could come directly to a movie; I searched for a long time to find a novel which had the model of time travel shown in Primer: a character can travel back to a specific point in time, and from that point until the point at which they travel back, there will be two copies of the same person. The other geeky thing about Primer is the meta aspect of its hero: The hero of the movie uses this power to repeatedly travel back to a party that he visited once, in order to play the hero at the party. Delightful!

I wonder whether time travel is the most popular form of philosophy (or a mixture of pop-science and pop-philosophy, if there were such a thing.) It poses the question “What if this was possible?” It’s not an answer; it’s not catharsis; it’s not an explanation; it’s an endless series of questions: what if time travel was possible? would you travel? why? when and where? what then? … I don’t know enough about the categories of philosophy to know which branch of philosophy asks these questions, or if there are people out there mulling over these questions. Pointed introspection at the personal level has been rewarding for me.

The actual name of the episode is “The Betrayal.” ↩

Post 3 - Yet Another Streaming Service

2025-07-28T00:00:00+00:00

“You know, I watched a great movie yesterday; it was called The Movie That Can Never Be Watched.”

“Oh, cinema-in-cinema. That sounds interesting. Where did you watch it, was it on Netflix?”

“No, I think it was on BagelsForLife.tv.”

“Oh, I don’t have an account on that one.”

“Tough luck! You’ll have to wait until the streaming contract for this movie expires in 4 years and it moves to yet another streaming service that you don’t have an account on.”

If you can relate to this story, you should keep reading.

Once upon a time, there were no movies. Then, someone discovered a method to record moving pictures, and someone else figured out that recording people doing interesting things and showing it to other people would be a great business. Thus began show business. Showing movies to hundreds of people at once was efficient, so people did that. If someone told you about a movie, you would be able to watch it for a short period of time, or if the movie ever got “re-released.” If the movie was still showing at a theater near you, all you had to do was ask your friend, “Where did you watch that movie?”, book a ticket, and watch it the next day. Simple.

Then, someone invented a television. Showing a movie to millions of people through a television was even more efficient, so people started doing that: They would ask TV channels to broadcast movies, or they would make tapes containing the movie available for purchase. Soon, a company decided that mailing these tapes to people would make people want to watch even more movies, because who wants to leave their house anyway? Then, someone invented the Internet, and everything became a hundred times easier and better in an instant: Pay for a single subscription, and you get access to every movie that has ever existed. Wait a few months and even new movies are accessible. Perfection.

The wheel of progress is never-ending though: Soon, the companies that were producing movies realized that while a few hundred thousand people would watch their movies in a theater, millions were watching it at home through a subscription service. The subscription service was making money from their content and they could not stand it: So, they decided to make their own subscription service: Every single content creator decided that their content belongs on their own streaming service. So, we got Prime, Plus, Protein, and Peacock. Content started getting fragmented and we were back to the age old question: “Where did you watch that movie?” “Yet Another Streaming Service,” would come the answer.

The “where is that movie streaming again” question has become a constant companion in my life. I am asking everyone, all the time. Services like JustWatch.com have tried to catalog this information, but in my experience, it either does not have data for the movie I am looking for, or the data is out-of-date. Even if you know which service has the movie, you may not be able to access the movie due to geographic restrictions, content restrictions, not having a subscription, etc. I have not kept count of the movies that I would like to watch but I have not been able to because of paywalls, the service not being available where I am living, or not having a credit card that the service would accept.

Shockingly, this has even bled into sports events: Sports events live on advertisements and the more people who watch a sporting event, the higher the advertisement revenue from that event. There is no point in locking down sports content, right? Isn’t this why cricket matches are always playing for free in the store windows of appliance stores in India? If people can watch an event for free, that’s good for the advertisers and great for the network that is broadcasting it.¹ Somehow, streaming services have broken this business model.

My pessimistic assumption is that TV networks spent too much (well beyond their means) on streaming services. Now, they had the infrastructure to stream content, but they were running losses. Put two and two together: the temptation of actual money from subscriptions was greater than the potentially higher advertising revenue they could get but were not seeing materialize.

There is another facet to this: The implementation of streaming video is not easy and requires a lot of optimization and clout.² Notwithstanding this simple realization, every single television and media company has jumped heads first into this segment with below-par results in most cases. This is obviously not going to work: The amount of optimization required to get glitch-free video streaming is too high for a new company, which has just landed the rights to broadcast a big event. The fact that a service lands a big event deal could often spell the end of its likability because that is the first time people actually use the platform; they are probably immediately annoyed by the video quality (bad) or the amount of buffering (worse), and promptly get off the platform once the event is over. The VC-spirited “get users first, revenue second, and profit third” does not work in most cases; it certainly does not work if you don’t have a great product when you acquire³ the users for the first time.

I have no idea what is going to happen; the total number of people who subscribe to streaming services appears to be constant to me: No one will pay for multiple streaming services; rather choosing to rotate between various streaming sites over the year. There is a counter-trend towards reconsolidation with ISPs and cellphone service providers now attempting to bundle network access services with streaming, convincing the user that they would be paying less for both as a bundle, than they would if they signed up to either of them separately. This might work for a while, but bundling rarely works long-term. Right? There is even the crazier business model of streaming services within streaming services: A big streaming service which allows people to watch content from multiple streaming services without leaving the site, through even more subscriptions.

Is this argument sound? Please poke holes in it! ↩
Netflix sends appliances that act as routers and content servers to ISPs which can be installed directly within the ISP’s network equipment, physically reducing the amount of distance that needs to be traversed by content: Netflix - Open Connect Appliances. (The specifications on this page for the actual appliance are cool to read.) ↩
Another marketing term. ↩

Post 2 - Games and Movies - Violence

2025-07-27T00:00:00+00:00

Early in 2024, I bought a gaming console. This was the first time I was in possession of hardware that was meant primarily to run games. Before this, I had played a few car racing games (using CDs on a Windows desktop). More recently, I played the indie favorite, Papers, Please, on my laptop and thoroughly enjoyed the game’s concept and the implementation. I had never played any other type of game before. So, Last of Us was a pleasant surprise: It really pulled me into the story and the character. Just as the friend who recommended the game to me said, the game was just a medium for telling a touching story, and it should not be treated as anything significantly different from the other mediums for telling a story: novels, poetry, music, movies, podcasts. After completing Last of Us, I played the second game, Last of Us Part 2 as well. The second game was distinctly different from the first one; I believe I had more fun playing the second game: It had more challenging enemies and scenes that were harder to get through. There was quite a bit of violence, because I was role playing as a character in an apocalyptic universe full of monsters and enemies. I abhor violence in movies: It is boring to watch, and very often, movies with violence lack a plot. Why was one form preferable to the other? Is it even different, or am I simply trying to explain away a personal preference? These were some questions that I have been thinking about for the past few months.

There is a dual track in my mind: Violence in games is OK; violence in movies is not OK. Not OK in the sense that it is boring, and intolerable. Did I mention that violence in movies is boring? It really is. Take a look at one or more of these trailers: Trailer 1, Trailer 2, Trailer 3, Trailer 4, Trailer 5. The names of these movies are besides the point. If you watched all five of these¹, you would notice some common themes: a lot of dark, smoky, dusty scenes; many SUVs that always seem to be drifting (effortlessly) on a straight road; a lot of men in soiled clothes; blood, gore, fighting; the quintessential running through traffic without getting hit by an automobile. All of these things are so over the top that whenever I watch something like this, I can not help but laugh at the sheer excess, the irrationality, and the ultimate stupidity of it all; how much can one suspend one’s disbelief? What is the point of it in movies like this that have no ostensible plot? The excess reduces the movie to a carricature of life: It shows us everything that we know to be impossible, and implores us to believe that these things could happen in the real world. No, they don’t happen in the real world. This is not just fiction.

I don’t know who started the oneupmanship of violence in movies. It has been on the rise in the past couple years. Superhero movies in Hollywood have always been like this; I had the misfortune of watching one of the Spiderman movies in the theater in 2017; I could not wait for the movie to be over: Nothing really happens in these movies, they are set in a mindnumbingly predictable sequence: the first 30 minutes or so is setup (i.e. introducing the villain and why they are going to do something bad), the next 30 minutes is some sort of search for the villain, then, the interval; followed by a mindnumbing 60 minute fight sequence, where everyone on screen will engage in wanton destruction. They call it entertainment.

If you find these movies boring too and were wondering why these movies keep getting made, its economics, as it often is. A quick scan of lists of top grossing movies on Wikipedia hints at this.

In 2022, 5 of the top 10 movies were “action” movies
In 2023, 8 of the top 10 movies were “action” movies (the top 7 movies were all classified in the “action” genre with “thriller / comedy / drama” being the subgenre)
In 2024, 6 of the top 10 movies were “action” movies (the trend seems to be fizzling out after a peak, because a few comedies and historical fiction movies made it near the top)

The money that these films make is clearly coming from viewers who like these movies enough to pay to watch them.² And it looks like volume does not lead anyone to getting bored either. Even if someone likes a genre, would they keep watching the same kind of content in that genre forever? I loved stand-up comedy for a while, but then the artists changed, the comedy was not as good anymore, the observations started feeling forced and banal (rather than weird, novel, and interesting), So, I gave up on it and moved on to other things. There is a “captive audience” somewhere.³

This sort of violence is completely different from the kind of violence that I saw in video games. Yes, video games have fighting scenes in dusty, outdoor settings. Video games also have to be creative: they can not just rehash the same fighting mechanic from before. The average movie director has more-or-less given up on coming up with new ideas, while video games are all about new ideas; new “mechanics”. In video games, you are not a passive consumer of violent content, you are playing a character, and the character is being often stuck in situations where violence is one of the possible means to escape the situation. Another possibility might be to use stealth and avoid confrontation, or to form an alliance to avoid violence completely. The possibilities vary wildly depending on the game that you are playing. As the main character in a game though, even the violence that is unavoidable and must be perpetrated in order to move the story ahead is more meaningful than the meaninglessness of SUVs ramming into walls and heros and villains throwing each other around like rag dolls, and (always) destroying the property of the innocent street-side vendor. In games, the fighting is a mean to get the player to feel one with the character that they are playing, to feel the dread of entering an unknown place, and then the surprise of a jump scare from an unexpected enemy, or to feel the exhileration of getting the strategy right, and completing a scene.

Last of Us Part 2 does this very well: The initial part of the game has a lot of fights, this can feel like a never ending quest to some. (I thoroughly enjoyed finding the “right strategy” to get away from villains without having to engage them in a prolonged fight.) But after this initial sequence, there are a few rewarding scenes where the main character lives a calm life. Tormented by the past though, the main character then chooses to get back into the fight. I really did not think that the main character would choose to leave everything. I was playing the game without any prior knowledge about the story arc. So, I had no idea where the game would end. What I thought was the ending of the game (the calm life on a farm for the main character) ended up being the middle. This is an experience which simply can not be conveyed through a movie. The run time of a movie is clearly displayed on the film certification that is displayed right before the movie begins; worse still, the interval right in the middle of the movie clearly delineates the first and second halves, making it impossible to obscure the passage of real time and confuse the audience into a state of false security.

The portrayal of violence is not going to magically reduce: In order to be a hero, you need a villain; in order for movies to be cathartic, the villain needs to be vanquished. Movies like Aadukalam, Vadachennai, Dark Knight Rises, Pulp Fiction, Fight Club, and Joker; TV shows like True Detective, Homeland, and Ozarks are fine examples in my opinion. These works employ violence as a means to an end. Movies that have been classified in the “action” genre of late have been made by people who have decided that everything (plot, settings, logic) is secondary to the portrayal of blood and gore. The grotesque and exaggerated portrayal of cinematic and fictional violence is the only reason the movie was produced, and in the creators minds, the only reason it will be viewed. This approach needs change.

There is hope: The current trend is reversing itself; there were a few non-action genre movies which made it big last year in the theater. The theater is losing much of its relevance. It is acting as a proxy for the likability of a movie only because the box office numbers are public, while the deals made with streaming services are private.⁴ There may well come a time when streaming services start publishing the audience numbers for their shows, making it easier to recognize the movies and shows that became popular after tanking in the theater.

Sadly, I had to watch them before posting this here because I decided long ago that I would not send out links to text that I had not read or videos that I had not watched to anyone. ↩
Technically, these viewers may not know what they are going to get but at least, I think we can safely say that they think that they will like these movies when they choose to pay to go to the theater. Right? (Not everyone at the theater is there just because everyone else wanted to watch a movie because there is nothing else to do inside a city anyway and this was the only movie playing.) ↩
Is there anything more disheartening than reducing people to this category? This is phrase that is quite common in consumer marketing; I picked up this term in a theater where an advertising company was advertising the opportunity to advertise during the interval to a “captive audience.” ↩
Speaking of streaming services, check out tomorrow’s post for my thoughts on those. ↩

Post 1 - Installing Linux is (Somewhat) Easier Than It Used to Be

2025-07-24T00:00:00+00:00

Late last year, and earlier this year, I started thinking about whether I could continue using Debian Stable as my primary Linux distribution. Another thought which had been in my mind for a while was that I should try a rolling update distribution, such as Arch Linux. I never made the switch because Debian Stable works pretty well: It is extremely stable because all the packages are at least 3 years old, with security updates managed by the Debian team. (I have not idea whether they backport bug fixes; I have not run into any bug whose fix I wanted, so this did not affect me anyway.) But recently, the Linux desktop world has been moving quite fast: Wayland and PipeWire are becoming the norm (replacing X11 and PulseAudio). Old tools are being rewritten in Rust (such as ripgrep, fd, alacritty, autojump-rs) and the rewrites are much better than their older counterparts. These were some things that pushed me towards testing out Manjaro, a rolling update distribution based on Arch Linux. This post is about the installation process, not the OS itself, because I have only used it for a couple days at this point.

Installing most Linux distribution has become quite simple, compared to a few years ago. I believe this is because of two things: First, mostly thanks to the Calamares installer project, the installer for various distributions looks nearly identical. Calamares is a generic installer project, and they seem to be determined to make the installation process simple.

Second, this was the first time I was installing Linux after having understood a fair bit about UEFI and the boot process, during the past months. (I was in a UEFI rabbithole for a few weeks.) Understanding UEFI makes the installation process easy: I was no longer scared about my computer not booting after an installation, because I knew that I would be able to recover from it fairly easily. Even if the installation goes wrong, I know that I can configure the firmware that implements UEFI (and runs before the operating system) to boot from another boot entry or from a Live USB.

By controlling the BootOrder stored in Non-Volatile RAM¹ (NVRAM), either through the UEFI configuration interface or even changing values from a Linux live USB. As long as the BootOrder is correct in the NVRAM, and the boot entry for the operating system (Boot0002, say) points to a valid EFI executable file on the disk, the system will certainly boot and get to the Grub boot selection screen.

The following diagram is the “short” version of my understanding of UEFI:²

I also started noticing some things during the installation process itself: Immediately after you select the partitioning and click Continue, the partition table is updated and the new partitions are created as you want them. Further, these partitions are mounted where they would usually be, but with a prefix. For instance, the boot partition is mounted at /installer/boot, the EFI System Partition (ESP) is mounted at /installer/boot/efi, and the root partition is mounted at /installer (or something like this: I did not record the command output during installation, and by the time it dawned on me that I should have, the installation was already complete. If you are going through a Manjaro installation, please tell me the prefix that you see.)

Once the installation is complete, before rebooting the computer and leaving the live USB environment, you can check the three changes that the installer would have made.

Updates to the NVRAM

The NVRAM is updated with a path to the new EFI binary. There is a UUID in the output of efibootmgr, and this is the UUID of the /boot/efi partition, which you selected in the partitioning step of the installer. For Manjaro, this is a binary file and not really readable, which is a shame. When installing Debian, the file is called shimx64.efi and there is a Grub configuration besides the file, which is a readable text file.

$ sudo efibootmgr | grep -i manjaro
Boot0002* Manjaro	HD(5,GPT,PARTUUID-OF-ESP,0x800,0xfa000)/\EFI\Manjaro\grubx64.efi

$ sudo lsblk -o uuid,partuuid,mountpoint | grep PARTUUID-OF-ESP
ABCD-EFGH                            PARTUUID-OF-ESP /boot/efi

$ sudo file /boot/efi/EFI/Manjaro/grubx64.efi
/boot/efi/EFI/Manjaro/grubx64.efi: PE32+ executable for EFI (application), x86-64 (stripped to external PDB), 4 sections

I think that the installer can do better here: I selected a 260 MB FAT partition which was already present on my disk and marked it with the boot flag and mounted it at /boot/efi. There were two issues:

First, it gave me a rather scary warning telling me that I had not selected an ESP partition that was atleast 300 MB and that I may not be able to boot into the system: This is incorrect. The ESP being at least 300 MB is a recommendation from Manjaro, not a requirement for the system to boot (if you update kernels a lot and don’t clean up the old initrd files, then the size of /boot can become a problem.) However, this has nothing to do with the ability to boot. I guess if there isn’t enough space in the existing /boot/efi the installation would fail.

Secondly, it did not tell me that I should mount the existing ESP at /boot/efi. If no partition’s mountpoint is set to /boot/efi at the partitioning step, I believe that the EFI binary will be written to the disk at /boot/efi, which is inside the /boot partition, and thus not something that the UEFI firmware will be able to boot from: The firmware that implements UEFI will boot from any ESP and ESP only supports FAT.

This would cause the OS to be installed and Grub, the Kernel, and initrd to exist in /boot and the system itself to exist in the root partition, but there would be no way to boot into it. This might be the reason for many issues that users face. I found a thread about it on the Manjaro forums, and I wanted to reply to it, but the thread is closed! So, users will forever keep running into this issue and thread and never be able to find the solution except through trial and error:

100% confirmed now. The missing esp flag is making me unable to boot Manjaro after install, it doesn`t even detect as a bootable system.

– Manjaro Forum

Grub, kernel, and initrd inside /boot

The other partitoin that you must have created in the partitioning step would have been the /boot partition. This partition typically has a copy of Grub, Grub’s configuration, the Linux kernel (vmlinuz-*.), and the initramfs image, which is a bootstrap system that runs right before the Kernel itself runs (if I am not wrong).

This would be a typical ext4 partition. So, it is easy to just mount it and check what is inside the partition. It can be mounted using applications such as gnome-disks if you don’t want to use the mount CLI tool.

Updates to `/etc/crypttab` and `/etc/fstab`

If you are installing Linux after 2020, you are hopefully using an encrypted LUKS partition for your root and home partitions, or you might be using a LUKS-based logical volume which contains both the root and home partitions, as this makes it easier to resize your partitions. Nevertheless, in both cases, Grub itself needs to know how to unlock the root partition. In modern Grub versions, I see these sort of options for the Linux kernel start up command:

search --no-floppy --fs-uuid --set=root UUID-OF-BOOT-EXT4-PARTITION

linux	/vmlinuz-6.12-x86_64 root=UUID=UUID-OF-DECRYPTED-ROOT-EXT4-PARTITION rw  quiet cryptdevice=UUID=UUID-OF-ENCRYPTED-LUKS-PARTITION:luks-UUID-OF-ENCRYPTED-LUKS-PARTITION root=/dev/mapper/luks-UUID-OF-ENCRYPTED-LUKS-PARTITION splash apparmor=1 security=apparmor resume=UUID=UUID-OF-SWAP-PARTITION udev.log_priority=3

I don’t fully understand the options here. Particularly, I don’t understand why the root= option is repeated twice. Once before cryptdevice= and once after. I guess only the second one applies? It is also interesting to see that it is “resuming” the Swap partition. What is that supposed to mean? I am planning to convert my currently unencrypted Swap partition to a cryptswap, which is a plain LUKS partition with a random key (because I don’t care about the data that is written there after a reboot). These are some things to look into later!

References

The following article and the UEFI specification helped me a lot in undertsanding how UEFI really works in practice:

Random Access Memory ↩
Mermaid source for this diagram: https://gitlab.com/-/snippets/4874203 ↩

Using LLMs Daily

2025-07-06T00:00:00+00:00

The software industry is moving towards a world where every engineer is expected to use LLMs more; the practice of keeping track of LLM usage frequencies and advocating for more usage will probably become the norm in a few years.¹ I don’t believe the hype; I think we’re in a fairly early state. If one looks at what is possible today, and the historical practice in this area to over-promise and under-deliver, it seems highly unlikely that significant changes are just around the corner. But every article about AI has this mandatory suffix: “AI can’t do that; at least, not yet.” I am not sure what all of that is based on. As a full-time software engineer, I have been unable to ignore these products completely. People in other professions should definitely consider ignoring it, even though there are vague reports of LLMs having some impact everywhere. This post is a summary of what I have been using LLMs for recently and where I think they are usable.

I did not use LLMs for real until early this year. I would occasionally use some bot as a toy, mainly for amusement. My favorite was to plug in queries such as why this movie bad. The response was an unnecessarily long essay, where the bot pretends to be thoughtful, and goes on to list a bunch of extremely mundane and generic reasons, which would apply to almost any bad movie. The same thing happened with most of the other prompts that I was using. Now that I have started using LLMs on a more serious note and on a semi-regular basis, I am starting to see where these systems are helpful.

This post will be provider-agnostic. A cottage industry has sprung up around comparing and ranking models, and how they perform on a variety of “tasks.” Another one has sprung up around comparing the products that are built on top of these models. Most of these are like performance benchmarks for traditional software: they have little relevance to using said software in real life. MySQL benchmarks look great on paper, and I have had a lot of fun using sysbench to prove that a single configuration change will make binary log processing 5 times more performant. However, when this change is applied in production, the results are underwhelming.

I remember reading about models being able to pass standardized tests, and win games like Diplomacy, in articles from a couple years ago; these seem like fuel for the hype cycle to me: Put your model up against 10,000 tests, publish the 10 that it does exceptionally well on. This post attempted to compare a few models on a single task. The main thing I took away from it was the pointless nature of the whole ordeal. Despite testing many setups, the author proved nothing because there was no way to reduce these models to systems that can be compared 1-1 against each other: If one model did better on one metric, another one would have done better on a completely different metric. There is a lot of subjective stuff happening inside the LLM; no one understands how to compare this behavior objectively.

What Works

Some things do work well when conversing with an LLM. Something that I have found useful is summarizing simple information and converting unstructured information into structured information (tables, JSON, YAML) For example, if you were comparing phones, and wanted the release dates, screen sizes, and prices as a table, most LLMs can create this easily. This works irrespective of the input and output formats; Supplying a CSV file and asking for a new column with some data, or any other kind of text processing that can be done using awk and sed will probably work. I have mostly used these on YAML files, which I come across periodically while dealing with configurations. These use-cases are not surprising. Spreadsheets are able to do a lot of great and magical things, and the conversational interface is probably an improvement for some users. The model’s ability to explain its methodology is certainly the most useful part.

One surprising use-case for me personally has been for talking about open-ended questions. I don’t expect a real or accurate answer from the LLM at all; I expect responses that point me to places where a piece of configuration or some code might be wrong. I am engaging in a conversation in order to figure out which way to go, what to test, which parts of the code to investigate further. This has been useful when dealing with new technologies, where I am unfamiliar with the standard patterns. By asking questions, looking into the suggested options, poking holes through what the LLM is offering, I was able to learn the underlying technology iteratively, and it helped me get to a good solution. I don’t think this process is faster, but I definitely felt that I was able to learn more than I could have if there had been no LLM at all and I was just trying a bunch of things to get to the same solution. In some senses, it is like talking to a peer about a problem that you’re working on. However, the solution’s quality is limited by two factors that gets lost in the AI-driven hype fog:

How much do you already know about the basics of the problem space? Even if you are not familiar with the specifics of the system that you are using, are you able to model the solution well enough in your mind that? When an LLM suggests something, are you able to verify that it is going in the right direction?
How much are you willing to trust the model? If the LLM suggests a solution, are you going to implement it as-is in production? Do you have other environments where you can test the solution? Do you have peers who can validate the solution before you accept it? Do you understand all the side-effects of the suggested solution?

I don’t think I could ever really trust a solution that comes from the model directly. I would like to run a sandbox environment and test what I am planning to do. Depending on the task’s complexity and the potential for making subtle mistakes, there might be places where the model output may be usable as-is (say: A script which only reads from an API). I believe LLMs works better as conversation starters than as solution finders.

Another use-case is to answer questions that have an answer which can be found in a deterministic manner but the software to do so does not exist yet: Why do some people prefer to use non-ISO timestamps in non-UTC time zones? Why is it hard to convert those times to something that is easier to understand? Not having to visit worldtimebuddy.com in order to find out what “CST” means would be great. Again, the questionable trustworthiness of LLMs rears its ugly head. A deterministic program would be easier to trust and use on a daily basis.

A variation of this use-case is one where I am willing to roll the dice: For instance, simple syntax questions where the cost of being wrong is almost non-existent: “How do I specify defaults in Helm templates?”; “How do you use getopts in Bash again?”; “Is it case-when or switch-whatever in this language?” … Mistakes are caught right away, if you have the decency to use a compiled language.²

What Doesn’t Work

There are many things that do not work well though. These are, in my mind, segmented into two experiences: as the generator of content and as the reader of AI generated ~~text that is unnecessarily verbose~~ garbage.

As a generator, for me, LLMs have not worked with anything to do with general Linux configuration. The mind boggling multitude of valid configurations, the various versions of software involved, an inability to really parse structured documentation (such as manual pages), and a general lack of “convention over configuration”, makes them a very bad hammer. There are as many ways to use Helm as the number of engineers using it. Understanding where a variable is being specified from inside an Ansible task is quite difficult. There are a variety of ways in which SystemD configurations are written, stored, and referenced. How Gnome does something is explained in documentation but this might change slightly from one version to the next, and these changes might not be rigorously documented anywhere.

So, I was never able to figure out some simple things, such as how Gnome’s session start-up really works, and whether I can control what services are started up along with the gnome-shell session. (I wanted the GSD Smartcard service to stop starting up automatically whenever I had a YubiKey plugged in; the LLM sent me on a merry-go-around for a while, asking me to edit various files and check the result by rebooting. Finally, I gave up and looked for StackOverflow-style answers from other people and found one that explained how to do it, and why the solution works.)

There is also the constant risk of LLMs making up non-existent syntax, or using non-existent packages and asking the user to install those packages. I don’t think there will be any huge “jump” here. It is simply harder to deal with a significant Ansible repository (with hundreds of roles, playbooks, group variables) than it is to build something new and self-contained.

My perspective on the value of being able to build something new is that it is a useful skill, but it does not happen frequently enough, so automating it is not prudent. In my limited experience, engineers rarely build completely new things. Most of the time, software engineers are iteratively improving existing software. When it does come time to build something completely new, it would be much better to start from a blank canvas rather than by making small changes to AI-generated base applications.

(Wasn’t the whole genesis of the failed experiment known as Web 3 that software engineers were bored with building small improvements to existing stuff and wanted to … start from an empty canvas and revolutionize the world (of finance)?)

Generating boilerplate (or unit tests) is cited as another common use-case. Even writing boilerplate code can be useful (if not fun). If the process of writing it once helps you improve your personal tooling, enabling you to do the same thing much faster the next time you need to do it, then writing boilerplate is a required step in the process of sharpening your tools. Tooling improvements compound: a change to your editor configuration that makes boilerplate easier to write will make many other things easier. It was by writing Golang’s table-driven tests manually for a month that I started thinking about snippets and eventually found packages that made storing and inserting snippets into files easier. Once I knew about them, I used snippets for many other use-cases. Another one I remember: I learned about visual block editing in Vim when editing some constant definitions in PHP; but this allowed me to deal with TSV files much faster than before.

Learning how to do it by customizing your tool is faster than what might be the knee jerk reaction if an LLM were close at hand: Open an LLM in your web browser, ask it to do some thing, and wrestle with it until it produces the right output. This is the classic short- vs. long-term time-knowledge trade-off (spending a bit more time today to gain some knowledge which will make doing everything faster months or years from now)

Being a reader of AI-generated text has been absolutely frustrating. Whenever you see a document with too many headings, with each heading having a short sentence that regurgitates the title, and a numbered list below that sentence, you know what follows is AI generated. These are often incredibly hard to read, because they drone on-and-on about obvious things. While most of what is generated is not wrong, it focuses on the wrong things; the generic things: the most important thing about text that is used to communicate an idea to another human being is understanding the constraints that one would have to operate under. Is there a reason that the latest re-org makes this idea the most important thing now? Is there a business reason that we can not use the right technology to solve this problem, requiring some hideous workaround? Is there a general feeling that fully-managed services are better for a company because we are not sure what the headcount of our team will be in 4 months? These are the hard questions people want to see answers to when they are evaluating your idea.

Reading bad AI-generated descriptions makes some of the descriptions I have seen before much better (but still not justifiable as “effective communication”):

TTSIA (or “the title says it all”)
Link to another page; without any explanation
- Bonus: that page does not have any explanation either
Link to a Slack thread; again, without any explanation

I used to believe that those were the absolute worst; but AI-generated descriptions are worse. Descriptions are for communication between people.

Another pattern that is catching on is putting task descriptions into an LLM, and submitting the code that is generated as-is for review. Most of the companies which build “coding assistants” are actively boosting this use-case, because they believe that it will really move the needle (for them). The products that I have used are nowhere near this level; they struggle to understand the file that I have open in the IDE in front of me right now; if they can understand a single file, they are not able to look for other files and understand the structure of the code; most are unable to generate a valid Git patch that will apply on a file that I submitted as input.

Red Lines

There are some red lines for me: areas in my life where I don’t see myself ever using LLMs.

I don’t want to use LLMs for my personal projects, or for writing notes or posts on this blog. I do these for fun and learning; the output is secondary. Debugging an existing Ansible role written by an LLM is substantially less fun than writing something from scratch, and iteratively building it into something useful. It is the latter that aids fun and learning
I would not use LLMs in any setting where the accuracy of the output can not be verified easily and immediately; the temptation to just believe the model and move forward is strong because the natural language output resembles what a very confident engineer would write. However, making decisions based on the summary of a report is worse than not using the report at all, and relying exclusively on your intuition
I wouldn’t use LLMs to plan trips or prepare itineraries. These are again things that I like to do, and even though there are a million small decisions to make during the planning phase, it is usually quite pleasant. And even if the places I end up going to are not the greatest, I don’t mind that either: Once again, the output is secondary to the process.

I have been meaning to write about this for a few months now, but I have put it off because I have not been able to come up with a general framework for the places where AI usage is acceptable to me. I don’t have such a framework, and I believe that it will be difficult to come up with one because the tool itself is not static, making it hard to pin down.

One of the existing problems with the Internet is that primary sources are behind paywalls, and there are way too many secondary and tertiary sources (news sites, commentary, commentary about the commentary, …) LLMs exacerbate this situation by reducing the cost of running a tertiary site to zero: Publishing the summary of an article which summarizes a paper that appeared in a scientific journal requires no effort anymore. There are sites that mirror StackOverflow’s content and use SEO to get their websites to the top of search results.³ LLMs will help these owners summarize questions and put a short blurb at the top, further helping them game the search engine into ranking them above the primary source.

LLMs will create a world where text abounds and no one reads that text anymore. There is no surprise then that everyone is migrating towards short videos; forms of media where a person is portrayed.

I am under no illusion that this boulder has gained enough velocity that standing in its way would be foolhardy. In my opinion, in software, it has become a requirement to use AI/LLMs at work. Not using an LLM leads to the question, “What is wrong with it? Which product should we buy a license for instead?”, leaving little room for opinion or choice. I believe that defining the situations where you don’t want to use an LLM, no matter how good it gets, is the starting point in getting used to this new tool. In the face of a developing new tool, as products and companies navigate towards its eventual steady state usage patterns, it is important that everyone thinks about their red lines before using the tool becomes the norm.

I am talking about companies which have not realized this yet. This is already a practice in many places, from what I have heard anecdotally. ↩
If you are using an interpreted language, you have much bigger problems – I have forgotten the source of this quote. ↩
Why do such sites exist? What is their economics? ↩

The Promise of Open Source

2025-06-14T00:00:00+00:00

Using open source software can be a fair bit of work. I emphasize that this is a choice, as it is entirely possible to install Ubuntu on your computer and use it for all your daily activities, without thinking about customization. Say you want to open multiple windows, and have them all take up equal space on your screen automatically. This is possible: You can use a tiling window manager. There is free and open source software out there which can do it. However, it is not being marketed anywhere by anyone: so you have to search for it. There are no automatic QA suites that run against it, nor is there testing to verify if each version installs without hiccups on a wide variety of supported operating systems. There is no such thing as a fixed release schedule, or quarterly OKRs, or a vision document. The project does not employ UX researches, who talk to users of other tiling window managers, or product managers that compare the project against its competition. The setup that the project author is using might be the only supported system. To me, these are minor annoyances when one considers the substantial promise of free and open source software.

I use a mix of open source and proprietary software on a day-to-day basis.

My primary phone is an iPhone running iOS (proprietary)
The gaming console in my house runs a proprietary OS, originally based on OpenBSD (open source)
My computers have some distribution of Linux installed (open source, but with proprietary drivers)
Some of these computers have Windows installed in a dual boot configuration¹
The secondary phone is running Android. The OEM no longer sends OS updates, so it is stuck on an old version of Android.
The e-reader uses proprietary firmware and a proprietary operating system that is based on the Linux kernel
The Internet router uses proprietary firmware written and maintained by the manufacturer of the router

… this sort of inventory is interesting. I understand that living completely on open source software is very difficult. I am not an ideologue about this: One must use the tool that fits one the best. If you want to use iMovie to edit your videos, or iCloud Photos to manage your photo library, then you should. I believe that this choice must be personal and intentional. The only criteria for me is one must not “go with the flow” and choose the path of least resistance.

Let’s say I was to live completely on open source software. Almost everything that is proprietary in the above list can be replaced given infinite time, effort, and cash. This does not appeal to me, and may not appeal to you either, but it is possible.

For instance: Recently, I attempted to install OpenWRT on an old router. It went horribly: The installation seemed to “just work” at first, buoying my hopes. A couple days later, I noticed that it was not actually working. Attempting to track my way back to the problem, I managed to soft brick² the router. Through some (unknown and random) combination of reset button presses and pulling-the-plug actions, I was able to at least get the router to boot, and fix the router by installing its stock firmware. The router is now usable. However, the recovery was not complete because the login page for the router still (inexplicably) redirects to the OpenWRT LuCI CGI page, even though OpenWRT has been apparently completely removed from the device. I don’t need that router to work desperately, and I was experimenting for fun, so I knew the risks. I can still clearly see myself half a decade ago in college, considering this move to OpenWRT; back then, I decided not to try it because I might lose the only device I had at hand. This is the jeopardy of playing around with open source. (I have chosen a somewhat extreme example here. The stakes are usually much lower when one is dealing with software; for instance, if someone is considering replacing the Microsoft Office suite with LibreOffice, then go for it! There are no downsides and it is a reversible decision.)

The ability to read and improve code is the big advantage with open source. I believe that this is overstated in most circles, and especially by the ideologues. “You should use Linux because all the code is open source. If you see something you don’t like, then you can go in and improve it!” This is something that I have heard many times in conferences. However, this is not possible in as an individual user of a software package, because even though the code is available, the expertise required to update the software that one uses is not easy to acquire. In projects that are not particularly well-managed, even having the expertise might not be sufficient, because the projects’ dynamics are not very friendly to outside contributors (see this excellent story.)

I can think of many projects that I use where I would like to contribute minor changes. I have been able to make meaningful progress only on a few of these, because the programming languages I know are limited. Once again, the interests and expertise of each user become the deciding factor. This promise does have benefits for all users though: If enough people use a software package, then a handful of them may run into the same problem, and one of them might have the expertise to fix the problem. If the problem is actually fixed, then everyone who uses the package benefits from the output. This fundamental “more than the sum of its parts” nature is the reason I think contributing to an open source project is rewarding.

In my case, I have ideas for the things I use daily:

Firefox: A web browser. I have wanted the ability to set a native keybinding for copying the link of the currently open webpage in the browser. I get by using the Copy-as-Org-mode browser extension, along with the extensions keyboard shortcuts feature. This setup works well enough, but I was never able to understand Firefox’ code enough to make a patch
GnuCash: A Qt-based GUI accounting application. I have run into a couple obscure bugs in my usage. Neither of them were major, I reported one of them back in August 2020, but I did not get a response. I could not independently figure out what part of the code they were happening in. Another change I have wanted for a while is a minor update to copy, to make some alert messages easier to understand, but even this apparently simple change has remained elusive to me
Notmuch: An application to index and search e-mail based on tags. I wanted a feature to write tag updates to a file. I managed to make sense of the (excellently written and very readable) C code and wrote a patch for this. The patch works and I have been using the patched version of notmuch for several months, but I never understood the tests enough to fix the failing ones, and submit a patch upstream. I will probably not bother with applying this patch when I upgrade to the next version of Notmuch
PaperWM: An excellent Gnome extension which has given me the confidence to truly abandon i3wm and stick to Gnome (I know Gnome has a lot of issues.) I ran into a serious, and distracting bug; one that a window manager should never have: 2-3 crashes everyday requiring a full restart! The bug was easy to track down, and a simple fix was easy to write. (This extension does not have any tests, so that was not a hurdle, but it also makes building confidence in a fix difficult.) I opened a patch upstream, and the patch was accepted into the version where I saw the error. The patch to the default branch was not accepted because I did not reproduce the bug on the latest version of Gnome. I totally understand the maintainers’ reasoning. After this positive experience, I feel confident that if I run into any other bugs with PaperWM in the future, I can fix the bug and work with the maintainers to get bugs fixed upstream.

Open-source legacy software also makes it more likely that a lighter weight replacement comes up and displaces the ancient behemoth. The classic case that I can think of is Wireguard’s displacement of IPsec. Wireguard was possible because the Linux kernel, and IPsec, were both open source, enabling the author to see the problems with IPsec’s huge size, and the possibility of a new VPN protocol that would fit well within the Linux kernel. The effort required to write and upstream Wireguard into the Linux kernel must have been huge, and it would not have been possible if any piece had been proprietary. A proprietary protocol might not be popular enough to warrant anyone thinking deeply about replacing it; or worse, the protocol might have become a commonplace due to its sustained usage at large enterprises, making its replacement technically challenging, and causing it to survive despite the existence of severe issues and better alternatives. I can not really imagine running IPsec at any point for a sustained period of time. Running Wireguard has not been a problem and I enjoy being able to run a VPN immensely.

Using open source software keeps computing interesting. It keeps enthusiasts engaged with the way computers are changing. It keeps users aware of what’s possible when one gets out of the walled gardens. There are perhaps three levels of engagement with open-source:

Users: Use things as-is
Hobbyists: Tinker with open source software at the edges
Enthusiasts: Make significant changes or new packages

Everyone is expected to walk down all three roads depending on what they use and their expertise: I am a user of VLC, Firefox, Debian as-is; I tinker with Gnome-Shell extensions; I build command-line tools to make configuring Wireguard easier.

Everyone can contribute, that is great. No one feels left out or like they are not contributing, that is even better. It is amazing that all of this, this ecosystem, continues to work. I will always be grateful for open-source software and actively support it by going down whichever one of the above three paths makes the most sense.

It has been more than 2 years since I last logged into Windows, so I truly wonder if I need it anymore. ↩
A state in which the router’s firmware and software is somehow corrupted and stuck in an inexplicable state of limbo. ↩

The Perfect Window Manager

2025-03-02T00:00:00+00:00

Over the past 10 years, I have tried off and on to stick to a single tiling window manager, i3wm. It has never really stuck on with my workflow: One of the biggest problems that I have faced is that while i3 solves one major problem (efficient screen space usage), it introduces several small ones (audio input/output selection, display selection, and many others). Others who are more proficient with the complete Linux stack face similar issues too. This post is part rant and part solution. The solution is to use PaperWM, a Gnome Shell extension that makes the simplest form of tiling possible: vertical tiling with window resizing. I have been using it for a week, and I am very excited about it, because it combines the battle hardened it will work guarantee of Gnome, with the features of a tiling window manager that I am most interested in.

For those who are used to dragging windows around with their mouse, the merits of a tiling window managers (WM) might be hard to understand. If you don’t have any problems with the way in which you arrange windows on your screen, then you probably don’t need a tiling window manager. The most important issue with my setup that irked me was that I usually have three windows open and a fairly wide 21:9 monitor which can comfortably accommodate them. The default solutions that I was using (Gnome or the default floating WMs on Windows or Mac OS) supported only three modes:

A floating mode, where windows can be placed wherever one pleases,
A single window mode, where a single window takes up the whole screen
A dual window mode, where each window takes up half of the screen

It seems arbitrary to limit the user to a single vertical split. Why do these solutions don’t extend this arbitrary limitation? The obvious one is that there is no need to extend the solution and everyone is fairly happy with one of the above 3 modes. The only other clue that I could find comes from one of the maintainers of Gnome’s window manager: It is not possible to have three column tiling without relying heavily on keyboard shortcuts.

I’m not sure how triple-tiling is expected to work? I don’t think we want a tiling system that only works via half a dozen obscure keyboard shortcuts, so how would mouse-based triple-tiling work without a screen edge?

– Triple-tiling for windows (#966) - GNOME/mutter

This seems like a valid reason to not build it into a window manager which only allows users to interact with it using a mouse. But it can be implemented if only the experts decided they wanted to or their users wanted it. This highlights (what I believe to be) the true nature of open-source: “Anything is possible. But only what the maintainers want is realized.” This is just how it is.

Options for Tiling Window Managers

There are several solutions:

I have used i3 and gone fairly deep into configuring it. I had reached a point where I was using and loving i3 daily. But only for a few days at a stretch. (More on why the streak isn’t longer later.)

I have tried AwesomeWM and Sway. First, Sway: I spent very little time with it. I was not able to launch gnome-terminal in Sway. I don’t know what went wrong, but digging oneself out of a hole where even the Terminal does not start is unthinkable.¹

Next, Awesome. I installed this one too and spent a bit more time in it. One thing that put me off a bit was that the configuration is in Lua. Another thing that surprised me was the concept of “Layouts”. These do not exist in i3. Layouts are predefined structures, that define where windows can be. The operative word here is predefined. i.e. It is not possible to edit them when using the window manager. They must be defined in configuration before one begins using the window manager. This reduces flexibility and forces me to specify how I will want to arrange windows now. I think that’s impossible and I will end up with a system that is less flexible and harder to use than a floating window manager. (One saving grace is that these window managers have a “floating” layout, so at least they are not worse than a floating WM in any situation.)

I have not used the other two. dwm sounds like a non-starter to me because there is no configuration at all; i.e. the configuration is maintained in a C header file config.def.h. It seems strange, unnecessary, and hard to use. I am sure there are good reasons for it. I have wished for well documented configuration in the common markup languages such as YAML, TOML, or JSON; but never have I wished to have configuration within code. Overall, there seems to be little documentation about dwm on its website, and there are no screenshots either. This seems like one of those (unnecessary but) intentional tactics used by projects to keep amateurs away. Though unrelated, I also saw this on Arch Linux’ installation guide, which is quite fitting with Arch’s image as a project for the expert tinkerers:

Boot loader: Choose and install a Linux-capable boot loader.

– Installation guide - ArchWiki

… The link does not tell you how to install a boot loader. I don’t like this attitude much. Software exists to serve people; it does not exist to force people into messing up their computers. (This is the anti-thesis of Debian, which exists to make operating systems open and more usable.)

xmonad is the one window manager that I am intrigued by. I cannot make up my mind over it. On one hand, I would like to try it out because so many people rave about it. On the other hand, it has layouts too, and the configuration is written in Haskell. (It is harder to read Haskell than to write Lua.) I will probably not try it out because I feel like there is no new concept in it, and it would be a solution looking for a problem in my case. I simply do not like this “layouts” thing. Speaking of:

Predefined Layouts

These window managers come with layouts which must be defined before usage begins. For example, one layout that is built into some window managers is “primary-stack” and it looks like this:

----------------------
|      |      |      |
|      |      |      |
|      |      |------|
|      |      |      |
|      |      |      |
|      |      |------|
|      |      |      |
|      |      |      |
----------------------

Here, there are two “primary” windows, which take up the full vertical length, and many windows within the stack, which take up the remaining horizontal width and appear on top of each other. Windows can be moved around freely. Layouts can be cycled through. The layout will stay in this 3 column type layout for the time that we use it though.

There are some basic limitations to this approach. They are best demonstrated by this question:

Does this layout exist? If not, can anyone suggest a good starting point for implementing it?

– Reddit - r/xmonad

The layout that the user is asking for is this one:

--------------------
|         |        |
|         |        |
|         |        |
|         |--------|
|         |        |
|         |  Tabs  |
|         |        |
--------------------

How can I define all the possible layouts that I might ever use until I come across every single thing I would like to do within the window manager?

I know that the retort to this is that these are flexible systems, which can be made to do anything that one wishes for; all one has to do is learn how to configure the system. My problem is with the approach, not the configuration. A system should bring flexibility without configuration. i3wm does this perfectly, by converting window management into a tree:

There is a root container
There are non-root containers
Each containers has one or more windows
Each container can be in a mode that the user chooses: tiled, tabbed, stacked
Windows can move freely between containers

This approach makes arranging windows in the layout that the author asks for in the above post trivial. The only learning curve the user has to go through is understanding i3wm’s excellent documentation and setting up keybindings so they know how to make the above choices.

Just Use i3, Then!

If you’ve gotten this far, you have read as I have defended and praised i3. It has a lot of things going for it: The project is very easy to build, has great documentation, and has a configuration file that is in plain text and rather easy to understand. The project brings a window manager, a status bar, and a lock screen, things any window manager needs.

But … the demerits have simply proven to be too many and too much. The breaking point was when I ended up being late for a meeting by several minutes because an external display had decided to turn off (power saving setting), and i3 does not fall back to the internal display of the laptop in this case. (i3 does almost nothing automatically. It is extremely thin.) So, I had to guess my way around a black screen: Use “Mod+Enter” to open a terminal, run a command such as xrandr --output eDP-1 --on to enable the internal display. After this, the audio devices were not switched (they were still sending audio to the display that was disabled, and I could not move that back to the internal audio within pavuctl.) This ended up frustrating me and was the straw that broke the window manager’s back.

The above problem is not new to me: I think most i3 users probably deal with it constantly. Similar things happen if you happen to lock your computer (using i3lock), walk away, and return in an hour or so when the external display has probably disabled itself to conserve power. Either you type something and i3lock’s splash screen comes up, or nothing happens, and you are just stuck: i3 does not reliably do one of these two. This uncertainty in state is hard to live with.

The other problem is with audio devices, their reliable operation, and the ability to switch between them easily. I have been using Linux as a personal device, which means almost no screen sharing or video calling, as most of that happens inside WhatsApp these days. So, Linux sound issues have only plagued me when I am attempting to run a browser within i3 and PulseAudio simply does not detect that an external display which is capable of playing sound is connected. This has also lead me to just kill the i3 session and switch to Gnome.

The final problem is Bluetooth. Once again, when using the computer as a personal device, this is not much of a problem. I have a setup with wired speakers always connected to the computer anyway. When I am outside my home and using the device, I usually go to Gnome because I want to use a Bluetooth audio device. (Audio and Bluetooth? Just give up.) The common solution is to use something like blueman. blueman some times simply does not work and comes up with arbitrary Bluetooth protocol errors which I know do not indicate anything because neither the Bluetooth chip nor the audio device have anything wrong with them. It is just software which is acting up.

I have lived with all this for so long, because I wanted to have three equally sized windows on my screen. This was particularly useful during development: editor, terminal, browser; or during writing: editor, browser, music application; etc. i3 is also flexible enough that I can keep changing the layout to the needs of the moment: If I would like a super small third pane, which only shows the music application, that is possible by going into i3’s resize mode. Switching from 3-column to 2-column or 4-column is trivial. But now that I have started using a Linux computer at work (for the first time in my career), on a day-to-day basis, this unreliability is not going to be tolerable. After setting up a bunch of applications and the layout that I want to work in, having to kill i3 and go to Gnome for a single meeting, and then, to go back to i3, and re-establish what I had set up before … That’s not good enough.

What Next?

This is why I feel like I have found the solution with PaperWM, a Gnome Shell extension. So far, things have been smooth. (Bear in mind that I have only used this extension for 2 weeks.) I will continue to watch out for instability and sincerely hope to find nothing.

PaperWM’s idea is to keep a train of windows, only a few of which are visible at any given point. Windows that go past the screen stay invisible. This is a brilliant solution to simplifying i3, without giving up on the most basic requirements of a tiling window manager. While i3 has to maintain a tree where each node is a windows, PaperWM only needs to maintain a linked list. This reduces the possible operations that are possible significantly! This is how using it usually looks like:

         -- VISIBLE  PORTION --
         ----------------------
|  W1  |  W2  |  W3  |  W4  |  W6  |  W7  |
|      |      |      |      |      |      |
|      |      |      |      |      |      |
|      |      |      |------|      |      |
|      |      |      |  W5  |      |      |
|      |      |      |      |      |      |
|      |      |      |      |      |      |
         ----------------------

Here, I have opened 7 windows. Of these, the windows W1 and W7 are not visible at all. W3, W4, and W5 are completely visible, while only a part of W2 and W6 are visible.

One additional feature is the concept of “useful widths” that one can cycle through using just keyboard shortcuts. Each window’s width can be set to 1/3rd, half, 2/3rd, or anything that the user might want. This is set in configuration. So, the user can decide what is useful to them.²

There will be bugs, I am sure. But even if there is a bug, this is a GNOME Shell extension, written mostly in JavaScript. I have no idea how GNOME extensions are written. But the project feels vastly more approachable, and easy to contribute to, compared to the main i3 codebase (or any of the other tiling window managers.)

Despite all the bad experiences, it seems highly unlikely to me that I will give up on i3. Recently, I have procured a spare laptop, that I intend to use over the next few months to test out a few different distributions of Linux (Manjaro Stable, Fedora, and Nix OS are the ones I have in mind), in order to keep things fresh. While Debian with Gnome and the PaperWM extension will be my daily driver for all activities where I expect things to “just work”, I will keep trying to find a solution that incorporates i3 and has all the peripheral applications that make a tiling window manager fun and easy to use.³

I believe that I ran into [SOLVED] How to get gnome-terminal to start in Sway? / Newbie Corner / Arch Linux Forums and the solution is to run dbus-launch gnome-terminal. I don’t know where to run this command if none of the terminals themselves start? Hilariously, I was able to run rofi, the application selector. ↩
This exists for heights too, though I have not used it. ↩
All i3 needs is something like gnome-control-center that is native to it. That’s my dream setup. ↩

Better Ansible Playbooks and Roles

2025-02-02T00:00:00+00:00

Ansible is one of those technologies that most SREs have come across at some point or the other. Despite the mindnumbing increase in abstraction layers in the past few years, trusty old Ansible does not go anywhere.¹ Ansible is based on a very simple idea: declarative management of the state of a computer. The implementation of this idea, and its final use-cases are fairly complex. Ansible is used for everything from setting up computers and servers, to deploying continuously to VMs. I have been using Ansible for the servers that I manage: DNS, RSS reader. I have also used Ansible for setting up my personal Linux computer. With both these approaches, I have found some limited success, but I am left with a list of pain points almost always. The pain points are indicative of the increase in complexity when it comes to setting up computers, installing software and keeping it up-to-date.

The point at which I started considering writing a post on this topic was when I realized that I had over 50 roles, spread across three directories. I know that a bunch of the logic in some of these roles is simply duplicated across various machines that I have wanted to set up at various points in time. Before diving in and attempting to cull the duplicates and bring everything back under control, I wanted to take a step back and write down what I want from the eventual set up that I am thinking about now.

What Do You Need?

Scope creep is rather common in engineering projects. What starts out as a minimalist RSS reader might end up containing a full-fledged XML parser. My approach to Ansible has been admittedly haphazard. At first, after seeing a playbooks-global monorepo at work, I attempted to keep roles in their own repositories.

This quickly gave rise to the problem which the monoreop is supposed to solve: Duplication and sharing. A lot of the work that Ansible is used to do at the beginning of system set up is routine and the natural defaults which all servers should have anyway. These included things like:

Don’t use the default user that is provided by the cloud provider
Disable SSH using password
Disable root SSH
Set the shell for all users except the one that you are going to use to /sbin/nologin
Enable a firewall
Stop systemd services that you don’t want: Why would a server want CUPS? What is avahi?

Instead of solving this problem properly, by looking through what was required and what was not, I took a shortcut and this was a mistake. I started using the ansible.cfg variable roles_path everywhere. Now, all my Ansible directories import roles from every other directory. Resolving this is easy, simply time-consuming. It is a good lesson to not take shortcuts; it is better to postpone the implementation of a known solution, rather than to workaround the problem using a suboptimal one.²

Naming

Ansible is very readable and imposes no structure on the name of roles. So, you might end up with a bunch of directories, each of which have tasks/main.yml files and names which are not exactly descriptive. If you develop things gradually, then there might even be roles which are either unfinished (half-baked ideas) or roles which are completely developed, but were never tested. Returning to a role’s files a few months later and trying to remember where the role is in its journey is hard and unnecessary. Here is a naming pattern that I intend to use:

under-construction.[role-name]: during development and testing
No prefix, [role-name]: one the role is stable and well-tested
deprecated.[role-name]: once the role is no longer used anywhere else

The role name itself should have its own prefix to clarify where the role is / can be used:

Roles with the under-construction prefix may be included in the appropriate playbook, but MUST NOT be visible on the main branch.
shared.[role]: can be used in multiple playbooks
[service].[role]: can only be used in the playbooks that match the file name glob [service]-*.yml

The [service]. prefix ensures that the monorepo can still work, while the roles remain scoped. I will also attempt to give each role a clear purpose and a clear set of variables that they need to have to run well. This idea seems simple and good enough for the elementary use-cases that I have in my mind.

Testing

Maybe Docker?

I saw this post about using Docker to test Ansible recently. It is a brilliant idea and I want to try it out. A single monthly run which attempts to set up a “container” and tests the output will work for stateless services such as a DNS server or an RSS reader. These are self-contained services whose setup can be tested fairly easily.

I still don’t know how the more complex services would work. If one were to use Ansible to set up a VPN server for instance, one would have to set up two containers, and then test that the two containers are on the same network which is not the shared network in Docker. This setup is too complicated and I don’t really want to think about it at the moment.

Another way to simply skip this stage completely is to not have any automated testing at all. Instead, to attempt to keep the setup up-to-date manually. This approach is not good at all and I don’t have much belief that it will work. (This is the approach that I was using until now: I would use Ansible to set up a host, then come back to the Ansible only when the host broke. The roles themselves would be broken by then, rendering it useless in attempting to fix the host.)

Internet Access

The ultimate variable might be access to the Internet. I have decided to give up on this one though: Even VMs in the cloud can have problems which are temporary and undiagnosable: Recently, I found that apt update did not work on a GCE VM in the asia-northeast1 region. I opened a Support ticket. When a person from Support attempted the same thing two hours later, the dpkg mirrors which were referenced by the GCE VM by default were already fixed. So, was what I faced a true problem? Yes. Will I ever be able to find the root cause behind it? No. So, it is a good idea to just assume that your hosts will have access to download files from any website that is up during setup time.

Now, if it just so happens, that the website were you are downloading files from was taken down or the URL of the file that you need to download changed, then, that is just bad luck. There is nothing one can do about such things except to rely on their intuition about what a role is supposed to do and use that along with the written down purpose of a role.

Conclusion

Everything I have mentioned in this post are just ideas. I will implement some of them over the next few days, and post an update here, as to how it goes.

Not to mention Shell scripts. ↩
sometimes. ↩

Adventures with Wireguard and MTU

2024-08-31T00:00:00+00:00

This past weekend, I wanted to set up Miniflux, a browser-based RSS¹ reader. I have been using it locally for the past two weeks and it has been extremely worthwhile. It has the dual advantage of always being online and the ability to categorize feeds, which helps me group the feeds that I want to read (blogs) from those that I want to skim (news). I did not want to set up a Miniflux instance which would have ports open to the Internet. I wanted the instance to be available only inside a private network; I wanted to use Wireguard to set up the private network. I ran into a problem that looks extremely simple in hindsight: The packets which were being sent on the Wireguard interface were larger than the Max Transmission Unit of some router between the VPS² and my laptop. This is something that I have not run into before. It was interesting to delve through the various layers of Linux’ networking stack. What follows is an account of my investigation.

The initial set up was very easy to get up and running.

First, create a Wireguard network with 3 peers: the VPS, my laptop, and my phone. Inside the Wireguard tunnel, the VPS was at 192.168.1.1 and my laptop was at 192.168.1.2. ping 192.168.1.1 on my laptop yielded the correct result; 64 byte ICMP³ packets were being exchanged between the two ends of the tunnel.

$ ping -c 2 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=19.4 ms
64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=6.22 ms

--- 192.168.1.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 6.215/12.810/19.406/6.595 ms

Next, start Miniflux using a docker container: The Docker container was listening on the host port 10443 (i.e. 127.0.0.1:10443). Locally, from the VPS itself, requests to the Miniflux endpoints / (which loads the login page) and /version (which prints the semantic version of the running Miniflux) worked well. (The running database container has a port but this port is not exposed to the host network.)

$ docker ps --format "{{ .Image }}\t{{ .Ports }}" | column -t
miniflux/miniflux:latest  127.0.0.1:10443->8080/tcp
postgres:15               5432/tcp

Finally, install and configure Nginx on the VPS. A single site was enabled with the following configuration:

server {
    listen 192.168.1.1:80 default_server;

    charset utf-8;
    location /healthz {
        return 200;
    }

    location / {
        proxy_pass http://127.0.0.1:10443;
    }
}

After restarting the nginx.service Systemd unit, I expected curl http://192.168.1.1/healthz to return an empty response with HTTP 200 status code. This worked. The request was logged in the Nginx access log. I tried curl http://192.168.1.1/. Here, the problem begins to make itself known: The request reaches Nginx and is logged in the Nginx access log. The request also reaches inside the Miniflux container⁴. However, no response is returned to Curl. In fact, Curl just keeps waiting forever. The request is handled in a few milliseconds according to Miniflux.

This alone was not too distressing. It indicates that Nginx’ proxy_pass directive is not working. I don’t expect these kind of things to start working immediately because there is always some minor misconfiguration, which can only be found in a second pass over the system. Also, there were many parts of the setup that I had not even thought about at this point: firewall rules on my laptop; firewall rules on the VPS; (perhaps) the hosting provider has a separate set of firewall rules for outbound traffic (similar to GCP?); (perhaps) my router is not set up to accept incoming TCP or UDP packets. I would have cycled through these theories … had it not been for this:

$ curl http://192.168.1.1/version
2.2.0

Requests to the /version endpoint on the same server along the identical network path worked; the response was coming from Miniflux.

One more confounding variable: Another member in this Wireguard mesh network was a smart phone. I attempted to load http://192.168.1.1/ from that device in Firefox, and the page loaded! It showed the appropriate login page that I was expecting to see.

Tethering this phone to my laptop using a USB cable and turning off WiFi, I noticed that on the 4G network of my phone too, the login page worked. Here is a matrix of what worked and what did not:

Device	Network	`/version` works?	Login page works?
Laptop	Home WiFi	Yes	No
Laptop	Cellular	Yes	Yes
iPhone	Home WiFi	Yes	Yes
iPhone	Cellular	Yes	Yes

So, what was going on here precisely? This was going to be something either very interesting and specific to my setup, or something fairly silly, such as a misconfiguration somewhere. Nevertheless, I was in for the long haul: A completely broken system is easier to fix than a partially broken one.

`iptables`

I have tried my best to stay away from the actual implementation and usage of iptables. I know about iptables and their usual use cases. I can even get away with reading rules, such as this implementation of a reverse proxy using only iptables (courtesy: GCP):

iptables -t nat -A PREROUTING -p tcp -m tcp --dport $DB_PORT -j DNAT --to-destination $DB_ADDR:$DB_PORT
iptables -t nat -A POSTROUTING -j SNAT --to-source $LOCAL_IP_ADDR

I don’t think I need to explain the above. It is pretty intuitive.⁵ iptables is a core feature of the Linux Kernel, but there are some quirks in using it “properly.” Most of the quirks can be demonstrated using the following diagram, taken from the Wikipedia page on iptables:

Is the text too small to read? You’re getting the point now!

Jokes apart, the design of netfilter and the frontend to netfilter, iptables, is pretty good (I have heard.) It makes everything possible: you can “mangle” packets, update where they come from, update where they go, prevent packets from entering the system, … and so on. All of it happens inside the kernel. Generally, I avoid iptables and use the simpler wrapper ufw (Universal Firewall). ufw adds rules to the filter table’s INPUT chain. By the way, ufw alone is not good enough if you are using Docker. See this well-known flaw when the two are used together, which is introduced due to the rules that are added to iptables when Docker is installed.

In this case, I had added only the default ufw rules, which deny all incoming and routed traffic, and allow all outgoing traffic. To take this out of the equation, I disabled UFW completely. UFW outputs logs whenever it blocks packets and these can be seen in the output of dmesg -wT. The packets that were supposed to make their way back from the VPS to my laptop were not showing up here. Interestingly, in the few minutes that the VPS did not have any firewall enabled, I saw several logs to Nginx from masscan and ivre-masscan. These tools are scraping the whole Internet in about 5 minutes: That is very cool. I have used only nmap for amateur port scanning (mostly to see whether sshd is actually listening on some port) and netcat to check for things like MySQL servers and SSH servers, both of which will write their version and the authentication plugin that they are using if you open a netcat connection with them.

Docker

Oh, right. Docker starts virtual bridge networks which are separate from the host network, but are connected together through the 172.17/16 subnet. The device docker0 takes every packet that is bound for this subnet by adding a route to ip route, and presumably, redirects the packet to the right container.

All of this is fairly good. I did not have many doubts here. The requests were clearly going through Docker and reaching the container inside it. I started the httpd container; and … it worked.

# On the VPS
$ docker run -d --name httpd -p 8080:80 httpd:alpine
88d95f2bce5bae6edbfde87ac4e57295038689790bf37cd26e65ddc04b9f79f7

# On my laptop
$ curl http://192.168.1.1:8080
<html><body><h1>It works!</h1></body></html>

Once again, the small response sent by httpd worked well, whereas the login page was still not working.

To completely avoid having to deal with Docker, I downloaded the static binary of miniflux from GitHub and ran it locally. Once again, local requests to the login page worked, while remote requests to /version did not work. I was stuck with the same problem even without Docker.

`ufw`

As I said, I did not want ports to be open on the VPS because there are large programs that are scanning all of the Internet a few hundred times each day. So, I had enabled ufw allowing only incoming UDP traffic on one port for Wireguard and incoming TCP traffic on another port for SSH. I did not expect these rules to be causing problems, and they were apparently not.

Completely disabling UFW did not make any difference at all.

Note: One quirk of UFW is the file /etc/ufw/after.rules. When UFW is disabled, the iptables which are added by ufw are removed. The rules which are added to iptables from the after.rules are not reverted. I guess this is because UFW does not know how to revert these rules.

$ sudo ufw disable
Firewall stopped and disabled on system startup

$ sudo iptables -S ufw-logging-deny
iptables v1.8.7 (nf_tables): chain `ufw-logging-deny' in table `filter' is incompatible, use 'nft' tool.

$ sudo ufw enable
Firewall is active and enabled on system startup

$ sudo iptables -S ufw-logging-deny
-N ufw-logging-deny
-A ufw-logging-deny -m conntrack --ctstate INVALID -m limit --limit 3/min --limit-burst 10 -j RETURN
-A ufw-logging-deny -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[UFW BLOCK] "

`tcpdump`

I had reached the point where I would have to resort to using tcpdump. tcpdump is a great tool for debugging network problems. I have used it a few times when working with Istio.⁶

With tcpdump, the toughest part is to get to the really useful packet dumps. For this, I began with the requests that worked: I chose the very simple /robots.txt response from Miniflux.

$ curl -vvv http://192.168.1.1/robots.txt
> GET /robots.txt HTTP/1.1
> User-Agent: curl/7.74.0
> Accept: */*
[snip]
< HTTP/1.1 200 OK
< Server: nginx/1.26.2
< Date: Thu, 29 Aug 2024 05:59:25 GMT
< Content-Type: text/plain; charset=utf-8
< Content-Length: 25
< Connection: keep-alive
<
User-agent: *
Disallow: /%

The TCP dump output on both sides of the tunnel was identical. It looked like this:

(vps) $ tcpdump -n 'port 50000'
[snip]
10:47.533724 IP vps.43448 > laptop.50000: UDP, length 96
10:47.533856 IP laptop.50000 > vps.43448: UDP, length 96
10:47.541738 IP vps.43448 > laptop.50000: UDP, length 96
10:47.541923 IP vps.43448 > laptop.50000: UDP, length 176
10:47.541966 IP laptop.50000 > vps.43448: UDP, length 96
10:47.543038 IP laptop.50000 > vps.43448: UDP, length 272
10:47.548850 IP vps.43448 > laptop.50000: UDP, length 96
10:47.548850 IP vps.43448 > laptop.50000: UDP, length 96
10:47.548923 IP laptop.50000 > vps.43448: UDP, length 96
10:47.554451 IP vps.43448 > laptop.50000: UDP, length 96

This is fairly boring. The only thing to notice here is that the length of all of these UDP packets sent by Wireguard is very small. (This fact was not remarkable to me during the investigation.)

Next, the request that fails. GET /

10:54.636017 IP vps.43448 > laptop.50000: UDP, length 96
10:54.636139 IP laptop.50000 > vps.43448: UDP, length 96
10:54.641641 IP vps.43448 > laptop.50000: UDP, length 96
10:54.641641 IP vps.43448 > laptop.50000: UDP, length 176
10:54.641707 IP laptop.50000 > vps.43448: UDP, length 96
10:54.644767 IP laptop.50000 > vps.43448: UDP, length 1452
10:54.644779 IP laptop.50000 > vps.43448: UDP, length 1452
10:54.644781 IP laptop.50000 > vps.43448: UDP, length 1452
10:54.644783 IP laptop.50000 > vps.43448: UDP, length 160
10:54.645059 IP laptop.50000 > vps.43448: UDP, length 1452
10:54.645069 IP laptop.50000 > vps.43448: UDP, length 1452
10:54.645198 IP laptop.50000 > vps.43448: UDP, length 864
10:54.650279 IP vps.43448 > laptop.50000: UDP, length 96
10:54.650279 IP vps.43448 > laptop.50000: UDP, length 112
10:54.657194 IP laptop.50000 > vps.43448: UDP, length 1452
10:54.657203 IP laptop.50000 > vps.43448: UDP, length 1452
10:54.657206 IP laptop.50000 > vps.43448: UDP, length 1452
10:54.657445 IP laptop.50000 > vps.43448: UDP, length 1452
10:54.657450 IP laptop.50000 > vps.43448: UDP, length 1452
10:54.869238 IP laptop.50000 > vps.43448: UDP, length 1452
10:55.317282 IP laptop.50000 > vps.43448: UDP, length 1452
10:56.181234 IP laptop.50000 > vps.43448: UDP, length 1452
10:57.877260 IP laptop.50000 > vps.43448: UDP, length 1452
11:00.840330 IP vps.44215 > laptop.50000: UDP, length 96
11:00.840525 IP laptop.50000 > vps.44215: UDP, length 96
11:00.848374 IP vps.44215 > laptop.50000: UDP, length 96

Immediately, I noticed that these length 1452 packets were the ones which were not making it to the laptop. As I was running tcpdump on both sides, I could see that the smaller packets of various sizes (112, 96, 864) were making it through perfectly fine and in the same order. However, the largest of these packets, of size 1452 bytes, were not making it through.

I continued running the same tcpdump command and now checked the request GET / on my iPhone with the WiFi network:

10:30.755011 IP vps.44215 > iphone.50000: UDP, length 592
10:30.755320 IP vps.44215 > iphone.50000: UDP, length 148
10:30.755660 IP iphone.50000 > vps.44215: UDP, length 92
10:30.763983 IP vps.44215 > iphone.50000: UDP, length 32
10:30.767831 IP iphone.50000 > vps.44215: UDP, length 1312
10:30.767844 IP iphone.50000 > vps.44215: UDP, length 1312
10:30.767847 IP iphone.50000 > vps.44215: UDP, length 112
10:30.767950 IP iphone.50000 > vps.44215: UDP, length 96
10:30.778499 IP vps.44215 > iphone.50000: UDP, length 96
10:30.809367 IP vps.44215 > iphone.50000: UDP, length 480
10:30.811026 IP iphone.50000 > vps.44215: UDP, length 1168
10:30.820942 IP vps.44215 > iphone.50000: UDP, length 96

Here, things got interesting, Despite being on the same network, the VPS was sending the iPhone smaller packets of 1312 bytes. At this point, I pretty much knew that something was off about these large packets and some thing was dropping them in the middle. I still did not know what or why though. Before we get there, a small detour through one of the other theories I had during my investigation.

Other theories

My home network is a broadband Internet connection, which uses IPoE (Internet Protocol over Ethernet). This is an improvement over PPPoE⁷ with the primary merit being that there is no PPPoE authentication server when sending packets outside. Packets are sent directly to the ISP’s router which is closest to my home. Packets are authenticated implicitly: The very fact that I am able to send packets to the ISP routers at a given port implies that I must be a subscriber of their service. It works remarkably well; with download speeds increasing at least 2X from not having to go to the authentication server repeatedly. IPoE is not very well documented (at least from what I could tell.) The only documentation I could find was an old PDF written by Juniper Networks.

Also, my ISP is also only capable of IPv6 over IPoE. They partner with another company, which provides IPv4 over IPv6 transfer. This is powered by the Dual-Stack Lite (DS-Lite) protocol. I don’t know much about these protocols except for a basic idea about what they are supposed to do. One of the disadvantages of using IPoE is that there is no static IP support anymore: I am on a local network which is managed by my ISP and all packets go through this partner before reaching the requested IPv4 destination. All of these things sounded suspicious; were hard to understand; and sounded very much like the root cause of my woes.

The main difference between IPv6 and IPv4 is the size of the header: 40 bytes vs. 20 bytes. If I want to send an IPv4 packet inside an IPv6 packet, the header size would now be 60 bytes per packet, I reasoned. So, the actual data that can be transmitted over IPv4 over IPv6 networks would be 40 bytes less than what Wireguard thought it should be.⁸ The packet size which was not transmitted (1452) and the one that was (1312) seemed to point in this direction.

Answer: Max Transmission Unit

Finally, I landed on MTU and started thinking about it. The only other context in which I had heard about MTU was that of WiFi networks. Changing the MTU of a WiFi network is supposed to make things faster by increasing packet sizes, and reducing the number of roundtrips on the high latency WiFi connection between a device and the router.

My router’s settings page told me that the MTU which was configured for the home network was 1500. The MTU which had been negotiated with my ISP was 1454. This number was uncomfortably close to the size of the packets which were being dropped. I looked around for more information about MTU and landed on the precise problem that I was facing: MTU blackholing. The server was attempting to send a packet that was too big for some link along the way. Whichever router rejects the packet, sends a special ICMP message back to the VPS, telling it that the packet size is too big, and that the data should be broken up into multiple packets. However, the VPS is either not listening for these ICMP packets or blocking them or something else: It does not act appropriately and reduce the packet size. Instead, it just keeps retrying the same packet until the underlying TCP connection eventually gives up, assuming that the destination is not listening anymore.

The solution was to specify MTU = 1400 in the [Interface] sections of the wg-quick configurations on both the VPS and my laptop. After restarting the Wireguard interfaces, I was able to confirm that the solution worked by requesting http://192.168.1.1/ using Curl and a browser. The TCP dump output was interesting too:

30:35.332857 IP vps.38130 > laptop.50000: UDP, length 32
30:36.054528 IP vps.38130 > laptop.50000: UDP, length 96
30:36.054726 IP laptop.50000 > vps.38130: UDP, length 96
30:36.060136 IP vps.38130 > laptop.50000: UDP, length 96
30:36.072759 IP vps.38130 > laptop.50000: UDP, length 608
30:36.072832 IP laptop.50000 > vps.38130: UDP, length 96
30:36.073642 IP laptop.50000 > vps.38130: UDP, length 1432
30:36.073653 IP laptop.50000 > vps.38130: UDP, length 1168
30:36.078799 IP vps.38130 > laptop.50000: UDP, length 96
30:36.078799 IP vps.38130 > laptop.50000: UDP, length 96
30:36.081931 IP vps.38130 > laptop.50000: UDP, length 176
30:36.081932 IP vps.38130 > laptop.50000: UDP, length 208
30:36.082073 IP laptop.50000 > vps.38130: UDP, length 96
30:36.082292 IP laptop.50000 > vps.38130: UDP, length 176
30:36.082369 IP laptop.50000 > vps.38130: UDP, length 176
30:36.083529 IP laptop.50000 > vps.38130: UDP, length 304
30:36.088139 IP vps.38130 > laptop.50000: UDP, length 96
30:36.088729 IP vps.38130 > laptop.50000: UDP, length 112
30:36.088859 IP laptop.50000 > vps.38130: UDP, length 96
30:36.089944 IP vps.38130 > laptop.50000: UDP, length 96
30:36.089975 IP laptop.50000 > vps.38130: UDP, length 96
30:36.094581 IP vps.38130 > laptop.50000: UDP, length 96

The largest packet which was transmitted was 1432 bytes, despite the Wireguard link’s MTU being set to 1400. The 32 byte difference is probably because of the 16 byte header for Wireguard packets and the 16 byte authentication tag at the end of the UDP packet:

20-byte IPv4 header or 40 byte IPv6 header

8-byte UDP header

4-byte type

4-byte key index

8-byte nonce

N-byte encrypted data

16-byte authentication tag

– Re: [WireGuard] Header / MTU sizes for Wireguard

Without the MTU = 1400 setting, I ran tracepath --mtu 192.168.1.1 and found that the MTU detected by Wireguard was 1420. I have a sneaking, unconfirmed suspicion that this is because of IPv4 over IPv6, and its additional overhead of 40 bytes per packet. The numbers don’t completely add up: 1420 does not work, 1400 works, but the overhead of using IPv4 over IPv6 is 40 bytes (Right?)

So, my plan to confirm this is to set up a VPS which has IPv6 connectivity and use the VPS’ public IPv6 address and check whether the automatically detected MTU works in that case.

One point of proof that I have for this theory is another Wireguard network that has a VPS with IPv6 connectivity. When using the VPS’ IPv6 public IP address, I can confirm that the MTU of 1420 works properly in that case:

$ traceroute --mtu 192.168.1.1
traceroute to 192.168.1.1 (192.168.1.1), 30 hops max, 65000 byte packets
 1  192.168.1.1 (192.168.1.1)  112.788 ms F=1420  112.901 ms  112.854 ms

This investigation was a rewarding one and even though it took me nearly 3-4 hours to find all the components that were in the path from my laptop to the VPS, it was completely worth it to understand one of the fundamental systems that makes the Internet work: Path MTU detection.

Further research

PMTUD (Path MTU Detection) is a very important protocol. ICMP messages of the type Destination Not Reachable should not be dropped. There are some RFCs with solutions to the problem of over blocking of ICMP packets. I am not sure how much they have been adopted though. Perhaps the confusion among administrators who think that all of ICMP is bad is that ICMP is also used for Ping and can be used to detect the IP addresses that have any sort of running host.
TCP has its own variable Maximum Segment Size (MSS) which is decided during the TCP handshake. We can use iptables (yay!) to ensure that TCP’s MSS is clamped to the value of MTU.
tracepath --mtu $destination can be used to find the actual MTU that is detected between any two hosts. This works only if PMTUD is supported by all the hops along the path from the source to the destination. I don’t know what the decided MTU is if PMTUD just fails.

Really Simple Syndication ↩
Virtual Private Server ↩
Internet Control Message Protocol ↩
I restarted the Miniflux container with LOG_LEVEL=debug to see access logs. ↩
Of course, I am joking. These two lines add rules to the nat table’s PREROUTING and POSTROUTING chain. The PREROUTING chain rule looks for TCP packets with the destination port set to some required value $DB_PORT. If the packet matches this rule, then we “jump” (-j) to the DNAT chain. This is a virtual state which does what we want: “It specifies that the destination address of the packet should be modified (and all future packets in this connection will also be mangled), and rules should cease being examined. It takes one type of option: –to-destination ipaddr[-ipaddr][:port-port] which can specify a single new destination IP address” (man:iptables(8)). The second rule is self-explanatory. (Again, I am joking.) ↩
It is slightly funny to me that I was complaining about the complexity introduced by usage of iptables when I use Istio on a weekly basis at work. (Istio is a long wild river: how to navigate it safely - IstioCon 2021 is a talk from a coworker in the Platform team.) ↩
Point to Point Protocol over Ethernet ↩
Wireguard publishes the size of their headers here; but this assumes that a packet has only the IPv4 or the IPv6 address. ↩

Using an iPhone Without Windows or Apple Computers

2024-08-29T00:00:00+00:00

I bought an iPhone 5 years ago. Admittedly, it was a strange choice: I don’t like proprietary software that one can not customize. I knew that iPhones work well only if one was willing to pay for subscription services like iCloud and Apple Music. The fights that Epic Games and Spotify have started in court over Apple’s attempts to block their native integration into iOS is well-known. But I bought one anyway, because I could not get an unlocked Android phone (back then) in Japan, and I certainly did not want to buy a locked phone, pay for overpriced cellular network coverage for 2 years before the phone could be unlocked and I could switch to a different carrier. So, an iPhone was the least worst option. Over the years, I have struggled with iPhone’s software to do basic things, such as writing a text file, transferring photos to a hard drive, understanding how exactly WhatsApp and other such apps store their pictures within the phone, optimizing to never go past the 30% disk usage mark because flash memory becomes extremely slow beyond it and makes the phone unusable, and using browsers like Firefox without the ability to install ad or tracker blocking plugins. This post is a summary of some of the things that I have learned.

One thing that is worth appreciating is that iPhone’s software is rather strange and has a lot of features, which apparently improve security. I don’t know enough to say whether that is actually the case. Nevertheless, most obstacles faced during the process of taking data out of the iPhone are related to intentional design choices which make interoperability impossible. Even in the cases where it is possible, various hoops need to be jumped through and the process is probably a nightmare for a layperson who just wants to get something done.

Photos And File Transfer

Just the past week, I started using KDE Connect. There is an application on the iOS app store which serves as the client, and the Gnome Shell extension GSConnect runs on the Linux host. Photos can be transferred in their original HEIF format over a local WiFi connection. The installation of both these components is very simple.

The transfer speed is very fast (the line rate of your WiFi network or the speed at which files can be written to your local disk, whichever is lower.) The only consideration is that the phone’s screen should not lock during the period that the photos are being transferred. I was able to transfer as many as 250 files in a single transfer. I can probably transfer up to a 1000 files, but I have not tried it out yet. (If you have tried it, tell me the hardware that you are using, and I will update this post with the information.)

The same approach can be used to transfer text files (more on this in the next section) or to control the cursor and enter text on your computer from your phone or connected device (not sure why this is ever required? Slideshow?)

KDE Connect works by starting a TCP/UDP server on a port on the host. (Usually the port number is 1716, but check your local installation’s output for the command ss -tulpn to find the actual port. The process name is gjs on Debian 11.) This port should be able to receive traffic on the local network:

$ ufw allow in on wlan0 proto tcp from 192.168.1.0/24 to 192.168.1.0/24 port 1716
Rule added

$ ufw allow in on wlan0 proto udp from 192.168.1.0/24 to 192.168.1.0/24 port 1716
Rule added

Note: One thing you should remember to do is to turn off Gnome Shell Extensions completely once you are done importing files. I have noticed that once a phone is paired, whenever both the phone and the computer are on the same network, they get connected automatically. I don’t want that to happen unless I am manually connecting for some reason. There is an authorization prompt when a phone is paired for the first time. But that authorization is a one-time process, and things such as controlling the cursor of the computer using the phone or entering text on the computer are possible once this process is completed. This is a minefield which is best avoided by turning off Extensions completely.

Old Process: Using Windows

If you are not able to use KDE Connect, the following process will work. It takes more time, requires a recent Windows OS installation, and lot of moving files around between Windows and Linux. I am leaving this here as a record of my old (painful) process.

About 3 years after buying the phone and taking pictures with it, I wanted to import photos to my hard drive and store them there, along with photos from my handheld digital camera. The motivation was mainly to store all my photos together in a single Shotwell database. An added advantage was that resetting the phone to factory defaults and starting fresh would speed it up significantly, increase battery life, and reduce overheating during charging.

There were many hurdles. The first few hurdles were fairly simple:

Importing required a Windows OS running the horrible Windows application Windows Photos. Everything is “implicit.” So, you have to plug-in your phone and hope that the device is detected and an “Import Photos” option shows up. Apple has also seemed to have stopped maintaining iTunes, an application that I used several years ago to transfer MP3 files to an iPod.
You can set the iPhone to import photos in the widely supported JPEG format, or in the original “HEIF” format, which was hard to work with. I set the option to import as JPEG as I wanted to handle JPEG photos rather than HEIF photos. (The reduced disk storage, which is a major reason for using HEIF, was not a big issue for me at the time.) But if you try to import too many pictures as JPEG, then the import process … goes kaput, for lack of a better phrase. There is no error message and the import process just stays stuck for a period of time and then fails with the super generic Import has failed. Please try again later. message. I think this must be because of the “import as JPEG” option because the conversion from HEIF to JPEG must be performed on the iPhone and it is probably too resource intensive to be completed on the iPhone within some timeout.
- Recently, packages like heif-gdk-pixbuf and heif-thumbnailer have made working with HEIF files on Linux convenient. About a year ago, Shotwell also started supporting HEIF file variants. You need to be on the latest stable distribution version to be able to use the latest Shotwell version though. I tried to build the latest Shotwell version on a Debian 11 VM, and it was … very hard. I had to give up because many of the shared libraries which need to be linked are not updated on older distributions.
Once you start importing files as HEIF files, the process works most of the time. I have found that it is unreliable if you attempt to import more than 200 photos at a time. So, I would go from the past to the present and select months to import.
Don’t make the mistake of using albums on the iPhone because this metadata is not exposed at all and will be completely lost as soon as you bring the photos out of the iPhone. So, I have begun doing all of my categorization exclusively inside Shotwell.
After importing the files in the HEIF format, they can be converted into JPEG using CopyTransHEIC. This is a free piece of software for Windows. However, the free version can only convert 100 photos at a time. So, you will have to select batches of 100 photos and convert them to JPEG. This is also mildly annoying when working with large photo collections, such as one that I worked with which had 5000 photos, everything from the first day the phone was bought to right before the import. In that case, I transferred the files back to Linux, put them in 50 folders each containing 99 files, and then went back to windows: Ctrl + A => Convert to JPEG => Wait => Proceed to next folder.

Welcome to using computers, 2024 edition.

Plaintext

Plaintext is the only truly universal file format.¹ A plaintext file written in the 1960s can still be read using the standard tools which are installed on every operating system, including mobile phones.

iOS is the exception that proves the rule, of course. The OS works tirelessly to confuse users into believing that files do not exist and that folders are a thing of the past. The recently added Files app is a pretty bad application and is slower than most feature phones from the 2010s. The recent changes on Google Drive have a similar motivation, in my opinion. This approach has also been a major part of why Gmail is so popular. It encourages users to maintain a huge set of files, which are not categorized well. The only way to get to the file that you want is to search.

First, coerce users into creating files haphazardly without thinking about where they should put a file. (“You need a Document? Just create it on the home screen of Google Drive!”) Then, once you have made this a habit, sell them recurring subscriptions to software that you now have to use to make sense of all the documents which were created.

The only solution I have found to this is the brilliant iOS application Subtext. It is a simple text editor, which does not need an Internet connection to start-up. (This is a problem with Notion, Evernote, and Fastmail; all of which I have used briefly.) Files created using Subtext can be synced to any computer using KDE Connect, the method that I described for photos.

Recently, I noticed that even Dropbox now has the ability to create a text file inside its app. Files created in the app are automatically synced to your Dropbox Cloud account and accessible from other devices on the same Dropbox account. This might be useful for you if you are unable to use KDE Connect.

SSH

Being able to SSH into a remote server running Linux from an iPhone is a pretty big advantage. This is not an absolute requirement and I have lived without this for five years. Recently though, I wanted to use Ledger to keep track of some expenses while on the move. I could use Subtext to write each expense in Ledger’s understated and simple format:

2024-07-20 Dinner
  Expenses:Food:Dinner  INR 450
  Liabilities:CC

After entering a few expenses, in order to validate the text file and balance the books, I would need to use Ledger’s CLI somewhere.

Terminus is my preferred solution for this. Terminus doubles as both an SFTP client (to transfer the text file to the remotely running Linux machine) and as an SSH client (providing the ability to run commands on the Linux machine.)

I did not plan this all out before leaving for my trip; and the whole idea began as a way to streamline the integration of expenses during the trip back into my “main” expenses file. So, while I was on my trip, I generated an SSH key on Terminus, added it to Digital Ocean, created a Linux VM there, SSH-ed in using Terminus, installed Legder and some other tools like ripgrep. The whole process was surprisingly smooth. So, credit to the Digital Ocean frontend team for making a website which was super easy to use even from mobile Firefox. (The website was not responsive or a complete redesign or anything, but it was compatible with mobile Firefox completely, and used standard HTTP elements which work well on all browsers anyway, rather than custom JavaScript-based monstrosities which attempt to validate input before it is submitted or use modals for input fields, and seldom work well on mobile browsers.)

In my opinion, it is even more universal than PDF, which still requires a PDF reader, is not easily searchable without one, and is impossible to edit without paid software. I have tried to edit PDF files by breaking them up into images, using GIMP to edit the images, and then putting the PDF back together, but the final PDF looked noticeable worse than the original one. Good software which can do native PDF editing such as Bluebeam costs $260 per year for a single user. A steep price to pay. Plaintext can be edited using Geddit, Subtext, Notepad++, Notepad, and or some other built-in text file editor on every operating system. ↩

Notes and Review - White Noise (DeLillo)

2024-06-12T00:00:00+00:00

Rating: 5/5

What if a novel was very funny? If every single sentence that came out of the protagonist’s mouth, only served to make him look progressively less important and more foolish than you thought? What if it was the author mocking the protagonist privately to the reader, without letting the protagonist in on the joke? That would be a great novel, and White Noise fits the bill. The protagonist is a man who walks around a college in long robes and dark glasses, because it makes him look important and unapproachable, because it gives him authority. He continues this pretense in his thoughts as well. The chaotic misinformation rallies that go around in the back of his car, as everyone is talking over each other, and no one is answering the question which sparked the conversation, are a treat to read. The level of ignorance is deliberately exaggerated to a comic level. I can’t wait to watch the movie adaptation. Meanwhile, here’s a review of the book.

I present this (unnecessarily) long thought that is ricocheting around the protagonist’s phenomenally empty mind:

The kitchen and the bedroom are the major chambers around here, the power haunts, the sources. She and I are alike in this, that we regard the rest of the house as storage space for furniture, toys, all the unused objects of earlier marriages and different sets of children, the gifts of lost in-laws, the hand-me-downs and rummages. Things, boxes. Why do these possessions carry such sorrowful weight? There is a darkness attached to them, a foreboding. They make me wary not of personal failure and defeat but of something more general, something large in scope and content.

When he wants to describe something, the best the protagonist can come up with is “something more general, something large in scope and content.” It is DeLillo’s skill that we do not see this as bad or lazy writing, but as the character’s inability to come up with anything really insightful. It is not much of a comfort that the protagonist spends almost the whole book in this illusion: that he is saying something profound, when what he is actually saying is most banal. That is to say, the whole book is incredibly entertaining. He uses this “something” technique in one other memorable occasion:

In the morning I walked to the bank. I went to the automated teller machine to check my balance. I inserted my card, entered my secret code, tapped out my request. The figure on the screen roughly corresponded to my independent estimate, feebly arrived at after long searches through documents, tormented arithmetic. Waves of relief and gratitude flowed over me. The system had blessed my life. I felt its support and approval. The system hardware, the mainframe sitting in a locked room in some distant city. What a pleasing interaction. I sensed that something of deep personal value, but not money, not that at all, had been authenticated and confirmed. A deranged person was escorted from the bank by two armed guards. The system was invisible, which made it all the more impressive, all the more disquieting to deal with. But we were in accord, at least for now. The networks, the circuits, the streams, the harmonies.

The various characters are the main reason I remained interested in getting to the end of the book. The ending is not a satisfying one; it does not tie anything up; I guess that is predictable because the story never ends (because consumerism itself has no end).

First, there is the protagonist Jack Gladney. I have poked fun at him already. His “brain noise” is peppered throughout the book. And the story goes in some bizarre directions. He is the anchor among all the strange twists and turns (some of which are boring). He remains locked into a mind numbing set of car rides between his house, the college where he works, and the store. Every time he goes to the store, he buys things he does not need. Whenever he is at home, is throwing away things that he does not need. He is a symbol: A symbol of the mindlessness that the corporations expect from everyone, perhaps? There is one final thing that he does which I must mention. While talking with his wife Babette, he says:

“We’re going to come through this thing all right,” I said. “Maybe stronger than ever. We’re determined to be well. Babette is not a neurotic person. She is strong, healthy, outgoing, affirmative. She says yes to things. This is the point of Babette.”

The most present character is one of the professors at the college, Murray. He is studying “American environments.” And is constantly talking about the “data” being strong in the grocery store. He is a first class weirdo, but somehow he is the only one Gladney trusts and talks with and really believes. On multiple occasions what Murray tells him about something surfaces during another conversation or in another situation. Murray might well be the author’s stand-in: Always at the ready to explain things that we (Gladney is the naive reader’s stand-in?) don’t understand. Here is a classic:

“This is the nature of modern death,” Murray said. “It has a life independent of us. It is growing in prestige and dimension. It has a sweep it never had before. We study it objectively. We can predict its appearance, trace its path in the body. We can take cross-section pictures of it, tape its tremors and waves. We’ve never been so close to it, so familiar with its habits and attitudes. We know it intimately. But it continues to grow, to acquire breadth and scope, new outlets, new passages and means. The more we learn, the more it grows. Is this some law of physics? Every advance in knowledge and technique is matched by a new kind of death, a new strain. Death adapts, like a viral agent.

Heinrich is one of the kids in Gladney’s house. He, Heinrich, is Gladney’s son from a previous marriage, and his mother has joined a cult as their accountant. Heinrich is constantly the most interesting and the most confusing of all the characters: He talks about the looming ominous thing when there is a manmade disaster near Gladney’s house and he really shines during this time: He spreads all kinds of doubts, speculation, and theories about what is going on and is very authoritative about it. He asks Gladney questions that throw him off his studied center; these questions dethrone him from the pedestal of authority which Gladney maintains at the college by wearing his robe and dark glasses. At home, he is vulnerable to Heinrich’s probing.

Babette speaks the least. She only has one really long conversation with Gladney. And she goes through it matter-of-factly, without revealing much about herself. She remains the most mysterious until the end; even though she is being seen and assessed and commented upon by everyone else, she never has her own voice. I have a lingering feeling that she wanted to escape, and that she was one of the saner ones in the house looking for an escape actively, rather than lamenting the apparent lack of one as Gladney is or studying this condition as Murray was.

That night, a Friday, we gathered in front of the set, as was the custom and the rule, with take-out Chinese. There were floods, earthquakes, mud slides, erupting volcanoes. We’d never before been so attentive to our duty, our Friday assembly. Heinrich was not sullen, I was not bored. Steffie, brought close to tears by a sitcom husband arguing with his wife, appeared totally absorbed in these documentary clips of calamity and death. Babette tried to switch to a comedy series about a group of racially mixed kids who build their own communications satellite. She was startled by the force of our objection. We were otherwise silent, watching houses slide into the ocean, whole villages crackle and ignite in a mass of advancing lava. Every disaster made us wish for more, for something bigger, grander, more sweeping.

The story is meandering, and this is not a novel that one would read for the story. One might read it for its commentary on modern consumerism. One might read it simply because it is the funniest book ever. To attempt a serious review of this book would be to dissect a great joke. A joke such as this one:¹

PLEASE NOTE. In several days, your new automated banking card will arrive in the mail. If it is a red card with a silver stripe, your secret code will be the same as it is now. If it is a green card with a gray stripe, you must appear at your branch, with your card, to devise a new secret code. Codes based on birthdays are popular. WARNING. Do not write down your code. Do not carry your code on your person. REMEMBER. You cannot access your account unless your code is entered properly. Know your code. Reveal your code to no one. Only your code allows you to enter the system.

Quotes

I shopped with reckless abandon. I shopped for immediate needs and distant contingencies. I shopped for its own sake, looking and touching, inspecting merchandise I had no intention of buying, then buying it. I sent clerks into their fabric books and pattern books to search for elusive designs. I began to grow in value and self-regard. I filled myself out, found new aspects of myself, located a person I’d forgotten existed. Brightness settled around me. We crossed from furniture to men’s wear, walking through cosmetics. Our images appeared on mirrored columns, in glassware and chrome, on TV monitors in security rooms. I traded money for goods. The more money I spent, the less important it seemed.

– p.99

I threw away picture-frame wire, metal book ends, cork coasters, plastic key tags, dusty bottles of Mercurochrome and Vaseline, crusted paintbrushes, caked shoe brushes, clotted correction fluid. I threw away candle stubs, laminated placemats, frayed pot holders. I went after the padded clothes hangers, the magnetic memo clipboards. I was in a vengeful and near savage state. I bore a personal grudge against these things. Somehow they’d put me in this fix. They’d dragged me down, made escape impossible.

– p.338

(Gladney is the one narrating in both those quotes.)

“What is a radio? What is the principle of a radio? Go ahead, explain. You’re sitting in the middle of this circle of people. They use pebble tools. They eat grubs. Explain a radio.” “There’s no mystery. Powerful transmitters send signals. They travel through the air, to be picked up by receivers.” “They travel through the air. What, like birds? Why not tell them magic? They travel through the air in magic waves. What is a nucleotide? You don’t know, do you? Yet these are the building blocks of life. What good is knowledge if it just floats in the air? It goes from computer to computer. It changes and grows every second of every day. But nobody actually knows anything.”

– p.173

The others who spend their lives believing that we still believe. It is our task in the world to believe things no one else takes seriously. To abandon such beliefs completely, the human race would die. This is why we are here. A tiny minority. To embody old things, old beliefs. The devil, the angels, heaven, hell. If we did not pretend to believe these things, the world would collapse.”

– p.366

“My trainer says, ‘Breathe, don’t think.’ He says, ‘Be a snake and you’ll know the stillness of a snake.’” “He has a trainer now,” Heinrich said. “He’s a Sunny Moslem,” Orest said. “Iron City has some Sunnies out near the airport.” “The Sunnies are mostly Korean. Except mine’s an Arab, I think.” I said, “Don’t you mean the Moonies are mostly Korean?” “He’s a Sunny,” Orest said.

This is one of several random paragraphs that appear as if they are unrelated codas to the chapter. ↩

Beyond 100 Megabits Per Second

2024-03-02T00:00:00+00:00

After moving to a new apartment recently, I made a contract with an ISP offering a best effort 1 Gigabit connection. At both apartments I lived in previously in Japan, an Internet connection was bundled with the rental agreement. As these connections had a maximum bandwidth of 100 Megabit, I never looked at my peripherals to see whether they were capable of Gigabit. When I bought a computer, I noticed (without much interest) that the computer’s motherboard was capable of Gigabit ethernet. When I checked my Internet speed with a Gigabit connection, I noticed that it was only about 93 Mbps. This post is the story of understanding my home network, the various bottlenecks along the way, and finally, going beyond 100 Mbps.

Internet in Japan

Before we delve into the exact setup and the various problems that I encountered during my setup, it will be worthwhile to explain the way broadband Internet connections work in Japan. In any Internet connection setup, there are two parts to the connection: The first part is the Access network which consists of all the wires which go from each subscriber’s home to the “Edge router” of the Internet service provider (ISP). The second part is the ISP’s own network, which peers with Internet exchanges, public and private peers to provide full route transit to users. In Japan, these two parts of the Internet connection are owned and administered by two different entities.

The access network is more-or-less completely owned by Nippon Telegraph and Telephone company (NTT), one-third of which is owned by the Japanese government. Japan’s constitution has special laws regarding NTT, which started as a state monopoly, but was later privatized in 1985. Recently, other companies such as KDDI have also started laying their own cables to users’ homes. However, NTT still provides the widest coverage across the country, through their product, known technically as the “Next Generation Network (NGN)” with the brand name “FLETS”.¹ The NGN is able to provide upto 10 Gbps of bandwidth to individual home Internet users.

This separation of the access network and the ISP networks is unusual. It is the main reason for the large number of internet service providers for broadband Internet in Japan. The market is extremely competitive and there are perennial discount campaigns which are run by companies to get users to switch companies. As the access network itself remains unchanged no matter which ISP you sign up with, switching to a different ISP is as simple as signing up for a service online, receiving PPPoE username/password information via e-mail, and reconfiguring your router on the appropriate day and connecting to your new ISP.

However, this ease of shifting companies also means that predatory practices among ISPs are common. The most notorious of these being the “contract termination” fees. Several popular ISPs have fixed term contracts, which run for 2 or 3 years. If a contract is terminated during this period, users are charged the contract termination fee. (Interestingly, there are discount campaigns in which the new ISP will refund the user the contract termination fee that was charged by their previous ISP.)

In technical terms, this separation means that the equipment used to connect to the access network is the same, irrespective of the ISP that you choose to use. It is a small black box, called the “Optical Network Unit (ONU)”:

The ONU has 1 port where one can connect a Fiber optic cable, which connects to the closest telephone poll, managed by NTT and connecting the ONU to the access network. On the other side, it has a RJ-45 Ethernet port which can be connected to the WAN port of your home network’s router.

Another peculiar technology which is (probably) only used in Japan for home Internet access is IP over Ethernet (IPoE). IPoE is a technology which I could not find much documentation about at all. Several ISPs in Japan have recently started offering IPoE in order to connect to the IPv6 internet. There are also IPv6 to IPv4 translation services, which are integrated directly with the ISPs, and allow users to use IPv4 websites from their homes (even though they do not have an IPv4 global IP address). I have not used this because my router does not support IPoE. However, the Linux kernel has native support for IPoE, which means that if I connect my computer directly to the Ethernet port on the ONU, I can use IPoE to connect to the Internet, without installing anything or having to configure a username and password. I have no idea how this works without authentication. The documentation that I was able to find suggests that the user’s location is inferred based on the physical cable which they use to connect to the ISP.

The primary method for connecting to the IPv4 internet is the age-old Point-to-Point Protocol over Ethernet (PPPoE), which can be configured using a username and password issued by the ISP. (IPoE bypasses the authentication step, which removes the user’s dependence on the PPPoE authentication server, which is apparently a common cause for slowdown during periods of congestion.)

Home Network

The rooms where I had set up the ONU and the one in which I wanted to use a wired connection were different, and quite far apart. Fortunately, the whole house was wired with Cat 5E Ethernet cables. This cable is capable of Gigabit speeds.

The rough schematic looks like this:

Each room was connected, by wires running through the walls and the floor, to a closet in the living room. On the top shelf of this closet, was a wall-mounted Network switch, which can be used in addition to a router, to provide wired connections in each room in the apartment.

A switch is a networking device which sits inside a network and increases the number of ports on a router. While a router sits between networks (my home network and the ISP’s network), a switch sits within a network and increases the number of computers that can connect to the network. A typical router has a single WAN (Wide-Area Network) port and multiple LAN (Local Area Network) ports. By connecting one of these LAN ports to a single port on the switch, any computer connected to any of the other ports of the switch will join the same network that is being managed by the router. Switches operate at Layer 2 (data link layer) and do not know about IP addresses. They only know about MAC addresses and send traffic to the appropriate destination based on the information that is contained in each packet.

The router that I am using is old. However, it is capable of Gigabit speeds. So, from the ISP until the router in my house, Gigabit speeds were possible.

Problems

The first problem I faced was with the switch that was installed in the house. It was a very old 100 Megabit 8-port switch (like this one). I did not really need the 8 ports in the switch because I was planning to use a wired connection in only one room. So, I wanted to connect the LAN cable which ran from that room to the top shelf of the closet directly to my router. However, this was not possible because the cables were wired tightly, with no “play” in them to allow extension outside the socket that was connected to the switch. So, I bought an RJ45 cable extender, which can be used to provide metallic contacts and connect two Ethernet cables head to head:

The other problem I faced was that almost all the peripheral devices that I had were devices which only supported 100 Megabit. This included a Raspberry Pi, which I am using as a local DNS server, an old laptop, which has a 100M Ethernet port, and a USB-A to Ethernet adapter, which was only USB-A 2.0. This problem was resolved after I bought a USB-A 3.0 to Ethernet adapter, which did not require any drivers to work.²

Note: I do not recommend this product as the USB to LAN adapter of choice. It is the TP-Link UE300 1Gigabet Ethernet to USB-A adapter. However, it uses the Realtek RTL8153 IC chip inside it and this chip appears to be unreliable and after just a few days of use, the adapter stopped working properly. It would start in the “connected” state with the LED light flickering, indicating that it was working properly. But soon after that, the LED light would stop flickering and the LAN connection would no longer be recognized by the OS. This problem with the RTL8153 IC chip seems to be common and a reason for the unusability of many Ethernet adapters and USB-C hubs as well.

Finally, I was able to get speeds higher than 100 Mbps on a wired connection:

I tested this setup on Windows, Mac, and Linux, at various times of the day. And the speed were quite good, ranging from 250 Mbps to 400 Mbps. Upload speeds were even better than download speeds, maxing out at about 500 Mbps. (I don’t upload anything at all, so this part of the symmetric connection is mostly unused in my use-case.)

Conclusion

This was a run-of-the-mill setup post. Practically speaking, the difference between 80 Mbps and 250 Mbps is not particularly remarkable; I took this more as a game to get to a setup with faster “theoretical” speeds. Along the way, I learned about the various USB versions and their incredibly confusing names; information that was useful to me during future setups, where I had to choose the right USB versions for the other peripherals. One of the other interesting rabbitholes that I went down while doing this setup was the nature of USB-C and why it is reversible and how it can do so much, and will probably be widely adopted.

(I have not talked much about WiFi here. Basically, my router is capable only of WiFi 5, which is capable of more than 100 Mbps theoretically. However, the latency of wireless connections is generally high, and the Intel wireless chip that is in the motherboard of my Thinkpad laptop has typical roundtrip times as high as 100 milliseconds. Speeds vary wildly depending on the number of other devices and the type of data that they are using. So, I am glad that I have a good wired connection.)

I have no idea what this name means. ↩
There are other USB to Ethernet adapters, and some of them require drivers to be installed to get them working. If you are using Linux, and the manufacturer does not distribute drivers for Linux, then you should stay away from such adapters. ↩

Look Up

2023-12-31T00:00:00+00:00

“Look up,” “Take it easy,” and “Keep it real.” What do these phrases mean? Their literal meaning is easy enough to grasp. But they are never used in the literal sense. They are touted as cures to our collective ills; metaphors for the actual processes which would make everyone less cynical and more attentive. I heard these phrases in the coda of an inane Hindi movie recently. The movie contained generic drivel about young people: the improbably rich MBA graduate in her 20s, the extremely hardworking gym trainer that luck does not favor, and the stand-up comedian who appears to be happy-go-lucky but is in fact hiding a dark part of his past. These characters are “finding their way” in the world; the typical plot of a “coming-of-age” movie. The lesson of this movie was to convince everyone to put their phone in a (stupid and futile) bowl, “look up,” and take notice of the world around them. One of the characters is told to stop stalking her ex-boyfriend on Instagram; “I don’t know [why]; I can’t stop.” It is ironic that it is this same character, a few minutes later in the movie, who recommends “keeping it real.” What was the great revelation?

Much has been said about social media. This much we know: It makes everyone feel worse about themselves; consume more; buy things they don’t need; idolize people they don’t know; envy those that they do; and follow things that no one should spend any time on. Yet, Instagram and TikTok remain popular (an understatement). 10 people sitting around a table in a restaurant, each staring at their phones, is neither unfamiliar nor jarring. Every free moment is spent that way anyway. Why?

I am not interested in the generational answer. The one that begins with “Hamare zamane mein, …“ (“In our time, …”) There was never a time when everyone did not complain about the young people of their time. It is bad form to appreciate change, after all. Writing, telegraph, radio, television, the Internet, email, and Whatsapp: Each brought a revolution that some adopted, most ignored, and everyone had to live with. The time one waits for a response has changed; the person waiting still remains unchanged. Do you agree?

Many potential answers are left to consider. One of them is that the boredom and boundaries of ordinary life are much to strict, and that is what everyone is escaping in the digital worlds that they inhabit all day. These tools provide access to facts that were never before assigned so much value. What was once scoffed at as caprice is now applauded in the comments-sections as spontaneity.

The viewer’s insecurity is monetized by the “influencer.” In the ultimate portrayal of irony, the selfsame Hindi movie which triggered this line of thought in me, has this prize line of dialogue in response to the statement “She is an influencer.”

And who is she influencing?

A cursory look at the advertising industry suggests that advertising on the Internet is completely ineffective and that the continued investment of actual money is a product of group think.

This effort to jump the fence is further reinforced by innovations that keep us inside a larger, encompassing fence. One where limitations are enforced by the service providers. The professional reviewers have portrayed this as a willing trade-off of freedom for convenience. “Stay in the walled garden, obey the platform’s guidelines, and everything will be great.” Have you heard that before?

Someone said this long ago: “The current iteration of technology has solutions for problems which were created by the previous iteration.” The quest to build that single super-app is essentially eternal.

Own your devices; find your own (open-source) tools; build your own process; do not depend on the whims of large companies;

Contributing influences:

Hindi movie (that shall not be named for fear that others might watch it out of curiosity)
This quote from Meghan O’Gieblyn’s essay Contemporaries from the collection Interior States:

Perhaps the essential appeal of the digital world is its capacity not to distract us from the present but to clench us in its maw. There is something hypnotic in its assurance that nothing lies beyond the day’s serving of novel minutiae. To leave this world, even for an hour, is to find yourself drifting uncertainly beyond the margins of the moment. Your mind begins to wander, or else you find yourself slipping uneasily into the past.
Sacasas’ work: https://theconvivialsociety.substack.com/

A Chronicle of Predominant Conversations (2023)

2023-11-06T00:00:00+00:00

A large number of articles, opinion pieces, blog posts, video essays, podcasts, television dramas, and movies saturate the information landscape. It is bad form to say that there is more out there than can be read by any one person. The dreaded information overload has arrived. A few acknowledge the existence of this swarm of multimedia. The majority beckon an Algorithm, entrusting it with the responsibility of collecting, filtering, and sorting them in the unknowable order that each component particle of the majority expects. Opting out is futile. Not knowing about something is superior to not knowing about its occurrence. What follows is a view of culture and society based on the contents of 3 issues of the WIRED magazine.

Suggestions of openness are reduced to the realities of great coffee in the lobby in less than a decade.

Yet OpenAI has changed. The nonprofit board might technically be in charge, but virtually everyone in the company is on the for-profit ledger. … It’s got product managers and engineers working constantly on updates to its products, and every couple of weeks it seems to ping reporters with demonstrations–just like other product-oriented Big Tech companies. I have visited virtually every major tech company in Silicon Valley and beyond, and not one surpasses the coffee options in the lobby of OpenAI’s headquarters in San Francisco.

Not to mention: It’s obvious that the “openness” embodied in the company’s name has shifted from the radical transparency suggested at launch.

– What OpenAI Really Wants | WIRED (Retrieved: [2023-11-06 Mon 16:43])

Product companies have a bright future, if they focus on engagement.

Roblox Connect, then, is an attempt to get people to just … hang out in Roblox, even when they’re not gaming. Hanging out equals more engaged minutes; more engaged minutes leads to better advertising opportunities and “bookings,” the purchase of virtual goods using Roblox’s digital currency, Robux.

– Soon You’ll Be Zooming in Roblox | WIRED (Retrieved: [2023-11-06 Mon 16:24])

… and if that engagement is tied to commercial information.

First of all, in search, people come looking for information. Over the past many years, you know, how we present that has dramatically evolved. But we are still trying to help people find the best information that exists online. Inherently, people are also looking for commercial information, and ads are very valuable commercial information, because they connect merchants and businesses, small and big, to users. None of that changes just because we are applying AI deeply. When we evolve search with generative AI, we’ll apply the same principles. It’s important to us to connect users with what’s out on the web, and we are working deeply to make sure that continues to work well. … We want to make sure users are consuming those sites. So I don’t think the core part of the experience will change. We will have a space for ads in a way that makes sense for users and particularly on commercial queries

– Sundar Pichai on Google’s AI, Microsoft’s AI, OpenAI, and … Did We Mention AI? | WIRED (Retrieved: [2023-11-06 Mon 16:44])

Creation myths allow protagonists to enter closed restaurants and make small talk with staff after they were dropped off.

I arrived at a restaurant to meet Moore and tugged at the door. Locked–it was one minute before opening. … When the hostess let me in, I gave her the reservation name. “The other guest is already seated,” she said. … That was impossible. The restaurant wasn’t open yet. … “She’s been here awhile,” she explained. … She’d been dropped off earlier and passed the time by making conversation with the staff.

– She Sacrificed Her Youth to Get the Tech Bros to Grow Up | WIRED (Retrieved: [2023-11-06 Mon 16:10])

Definitions are changing.

I am a good mother. I define motherhood in much broader terms than just giving birth.”

– She Sacrificed Her Youth to Get the Tech Bros to Grow Up | WIRED (Retrieved: [2023-11-06 Mon 16:40])

Podcast-ready essays read by AI-generated voices.

The 19-year-old speaks in the confident, podcast-ready tone of a Silicon Valley entrepreneur who sees the world in terms of problems to be solved, ending every other sentence with, “Right?” Listening to him wax on about defensible moats and the “S-curve” of societal growth, it’s easy to forget he can’t legally drink. But then, occasionally, he’ll say something that reveals the wide-eyed undergrad, open to the world and still figuring out his place in it. Like the time he and a friend walked around the Santa Monica pier until 3 am, “talking about what we value.” Semrai thinks a lot about how to find balance and happiness. “I think, while I’m young, it probably lies more in exploring the derivative,” he says, “chasing the highs and lows.”

– The AI Detection Arms Race Is On | WIRED (Retrieved: [2023-11-06 Mon 16:41])

Profit still matters.

For a staunch advocate of the private sector, L also let slip flashes of exasperation at how corporate corporations can be. … Being on the same side of a war, it turned out, was no assurance at all of being in sync.

– In the War Against Russia, Some Ukrainians Carry AK-47s. Andrey Liscovich Carries a Shopping List | WIRED (Retrieved: [2023-11-06 Mon 16:03])

The masters spoke. They said, “AGI is not on the way.” No one was listening.

Now, Microsoft has an exclusive license to commercialize OpenAI’s tech. And OpenAI also has committed to use Microsoft’s cloud exclusively. In other words, without even taking its cut of OpenAI’s profits (reportedly Microsoft gets 75 percent until its investment is paid back), Microsoft gets to lock in one of the world’s most desirable new customers for its Azure web services. With those rewards in sight, Microsoft wasn’t even bothered by the clause that demands reconsideration if OpenAI achieves general artificial intelligence, whatever that is. “At that point,” says Nadella, “all bets are off.” It might be the last invention of humanity, he notes, so we might have bigger issues to consider once machines are smarter than we are.

– What OpenAI Really Wants | WIRED (Retrieved: [2023-11-06 Mon 16:55])

The optimists have abandoned ship.

“Artificial intelligence, cloning, genetic engineering, virtual reality, robots, nanotechnology, bio-hacking, space colonization, and autonomous machines are all likely coming, one way or another. But we must take a stand and insist that human values are folded into the development of each and every one of them.” Only a few years later, here he is rejecting not just these technologies, but technology writ large as a solution to our problems.

– Doug Rushkoff Is Ready to Renounce the Digital Revolution | WIRED (Retrieved: [2023-11-06 Mon 16:35])

Humans communicate to win esoteric games that simulate a world without bureaucracy.

This is the current Diplomacy metagame. Game theory calculations undergird most utterances, and even humans communicate in code. Lerer joked that in modern-day online Diplomacy, even human players wouldn’t pass the Turing test. Before Cicero, it seems, humans had already started playing like AIs. Perhaps, for an AI to win at Diplomacy, Diplomacy had to become a less human game.

– What If the Robots Were Very Nice While They Took Over the World? | WIRED (Retrieved: [2023-11-06 Mon 16:44])

Writing is still hard for the writers (apparently).

According to Thomas Mann, “A writer is someone for whom writing is more difficult than it is for other people.” “You search, you break your heart, your back, your brain, and then–only then–it is handed to you,” writes Annie Dillard in The Writing Life

– The AI Detection Arms Race Is On | WIRED (Retrieved: [2023-11-06 Mon 16:24])

Some things have not changed.

It’s a subtle but meaningful distinction: Human writing may not be better, or more creative, or even more original. But it will be human, which will matter to other humans.

– The AI Detection Arms Race Is On | WIRED (Retrieved: [2023-11-06 Mon 16:30])

Review - God, Human, Animal, Machine (O'Gieblyn)

2023-10-09T00:00:00+00:00

Rating: 5/5

I wanted to read a book which would delve into the philosophical underpinnings of the ongoing AI hype: What was the root of the idea that humans, with all their confusions and complexities, can ever be replaced by computers? O’Gieblyn’s column in the Wired magazine, Dear Cloud Support, is my favorite part of the magazine. Her writing is lucid and her references come from far and wide. I was not really prepared for the philosophical depth that is on display in this book: O’Gieblyn goes to the very beginning of the world and starts with the earliest philosophers (Plato, Aristotle) and ushers the reader through a series of “frames of mind.” She is not averse to religion or science; nor is she biased to any particular philosopher or their ideas; quoting from a huge variety of sources throughout the book to show the ways in which thinking has evolved. Her religious upbringing and her current vocation as a technology writer feature heavily throughout the book. After reading this book, I have a clear idea of where the foundation of the hype lies and how it has gotten this bad.

History of Ideas

Ideas do not just come out of nowhere; they are genetic, geographical.

– p.133

The concept that acts as the overture for this book is the Cartesian Partition: Descartes’ claim that the body and the soul were separate, that the body was machine-like, whereas the soul was responsible for “human characteristics,” symbolized by the internal experience, of a person. Descartes did not say much about the soul, or why it was supposed to be responsible for this internal experience. We can interpret this as his desire to analyze what he could and set aside what was intractable. Descartes’ aim with the partition between mind and body was to refute the leading theory of the time, Aristotle’s teleology, a doctrine which claims that ends are immanent in nature (i.e. vitalism.)

Aristotle’s teleology enchants the world with vitality and “ends” which are inherent to the object. (A stone falls to the ground because it is the nature of a stone to seek the center of the Earth, and so on.) Descartes’ cold partition of ends from the interior position that they occupied disenchanted the world, and made it susceptible to analysis. While the “soul” side of Descartes’ partition remained intractable, it was still possible to analyze the “body” side of the partition. This gives rise to modern science: Newtonian physics, computers, and information theory, which holds that information is capable of existing alone without any physical form at all.

From here, the threads of thought get quite entangled. First, on the “soul” side of the partition, there were attempts to resolve the hard problem of consciousness. There was Conway’s Panpsychism, that consciousness is everywhere and not very unique to humans at all. Through Leibniz, this philosophy undergoes a transformation and takes the mathematical form of the Integrated Information Theory (IIT) proposed by Goff and Tononi. The IIT proposes that consciousness is the result of a physical system that has a high degree of integration. The similarities between IIT and Panpsychism were not missed, because using IIT it is possible to prove that a small system of inactive logical gates is conscious, even though that makes no sense. Here we sense a vulnerability which is now being exploited by the peddlers of hype: No one is quite able to define what consciousness is or what it enables or how to even identify it. IIT remains widely contested and there is no agreement about whether it is the “theory of everything of consciousness.”

On the “mind” side of the partition, things seem to be going swimmingly. From Newtonian physics, we get many useful tools, the steam engine and computers being major among them. Newtonian physics’ focus on things (unsurprisingly) gives rise to Materialism, the idea that matter is the fundamental substance in nature and that everything springs from it. This was an idea that the German philosopher Hegel was affected by. Marx and Engels began their theory of human history mainly as a critique of materialism, ending up with a powerful criticism of Capitalism itself. Though, that is a topic for another time. Modern materialism is also the root for other things such as secularism, and alienation induced by modern economic systems.

The rise of materialism gives birth to the (frankly absurd) concept of Emergence, the idea that consciousness will emerge out of a system that is sufficiently similar to the human brain. O’Gieblyn recounts experiments which test this: Scientists who make robots and wait for them to become conscious. This was the most interesting part of the book for me. From the point of view of working with computers, this seemed like an unhinged effort. Imagine a mother board with a basic CPU, main memory, persistent memory, input devices without any software at all; i.e. Not even firmware. While that is an electrical circuit through which current will flow in some predictable manner, there is absolutely no way that the motherboard would behave anything like even the most rudimentary computer. O’Gieblyn sharply points out the contradiction: Emergence comes from the “body” side of the Cartesian partition, so, to claim that it in-turn gives rise to the “soul” side, which was intentionally set aside, is not possible.

Materialism spawns other ideas too. Dualism, succinctly summarized in the book as the idea that “mind is software that is running on the brain’s hardware.” This one also assumes that whatever the mind (or consciousness) is, must be part of the brain’s physical make-up and not something outside of it at all. Another stray idea is the mechanistic brain, or “machines can be conscious because consciousness is a mechanistic process, and we will figure out how to replicate it in machines.”

However, this apparent utopia with material on top does not continue for long: Challenges appeared from everywhere. Kierkegaard’s objection was that subjective experience is the only thing that matters; and his philosophy, lightly touched on in the book, was almost completely told through the first person without any affectation of a third-person point of view. Kierkegaard’s descendant is the famous scientist Bohr, who compared physics and poetry, saying that poetry was the task of creating images through the severely limited medium of languages, just as physics used languages and systems of reason to describe the world in an approximate sense; never being able to grasp everything at once. (Experts say that it is hard to classify Kierkegaard’s philosophy. So, I am not sure on which side of the partition he put more emphasis. But a rough guess would be that he preferred the soul far more than the body.)

Returning to the “soul” side of the partition, Panpsychism gives way to Idealism: the concept of a single unified mind which is the universal substrate of every existing mind. There is no Archimedean point, from which everything can be observed and commented upon. This concept also gives absolute primacy to subjective experience because that is the only way of describing or learning more about the world.

Kierkegaard insisted on the value of subjective truth over objective systems of thought. Fear and Trembling was in fact written to combat the Hegelian philosophy that was popular at the time, which attempted to be a kind of theory of everything—a purely objective view of history that was rational and impersonal.

– p.133

Metaphors

A large part of this book is about metaphors and how they are abused. While we used to say that computers process information like a brain does, now the metaphor is reversed and we hear sentences like “I am still processing that information” or “What they said did not compute for me.” This de-anthropomorphisation of human beings is insidious because it brings people and machines closer together without clarifying why.

Repeatedly, we see that what was a metaphor suddenly become the way things are talked about. All of science is a metaphorical way of explaining how things work to others and ourselves. However, if we were to believe that science itself was not a metaphor describing how things are but the absolute system which makes things the way they are, then we go down the slippery slope of granting more primacy to a system of symbols over reality itself: making claims that information and ideas can survive on their own forever without any subjective experience.

Hype

Remember how the words consciousness and intelligence seemed to lack a clear definition? Everyone was sure about what consciousness looked like (after all, we all have it); but no one was quite sure how to recognize it in something that was not human, or maybe even inorganic altogether. This is the vulnerability of the language which the con artists use to get into the news:

Such aspirations necessarily require expanding the definitions of terms that are usually understood more narrowly. If “intelligence” means abstract thought, then it would be foolish to think that plants are engaging in it. But if it means merely the ability to solve problems or adapt to a particular environment, then it’s difficult to say that plants are not capable of intelligence. If “consciousness” denotes self-awareness in the strongest sense of the word, then nobody would claim that machines have this capacity. But if consciousness is simply awareness of one’s environment, or–as has long been the case in artificial intelligence–the ability to behave in ways that appear deliberate and intentional, then it becomes more difficult to insist on it as a phenomenon that is unique to humans and other animals.

– p.109

It is perhaps appropriate to mention the strange case of an engineer thinking that an AI system was sentient. This engineer explicitly said that they were talking about sentience as a “priest” and not as a “scientist” It seems hard to interpret that sentence: What is the difference between a priest and a scientist? Is consciousness for one different from consciousness for the other? This latter question we can answer with certainty: Yes, consciousness is different for every single person and profession because there is no clear definition of it anywhere. But what about the former question about their perceptions of consciousness. If a priest can claim that something is sentient, is that a higher or lower bar of evidence than a scientist. This is the rub: Traditional modernity has lionized Science to the point where the S is capitalized; it would be foolish for a modern mind to claim that the priest’s point of view was more valid than the Scientist’s. But sentience itself is a concept so far removed from science, that it hardly matters what the profession of the person making the claims is. This is the point which is always missed: Everyone focused on the credentials of the person making the claim while the person making the claim was busy saying that they were not using their credentials to make the claim in the first place. It is clear how this situation can be quickly monetized for a modern audience that is not interested in the claim or the philosophical backing of it. They see only the hype itself, and start sharing the story for its own sake; for the absurdity that it represents, rather than for the truth content of the story.

Arendt was quite concise in expressing this emotion:

For Arendt, the problem was not that we kept creating things in our image; it was that we imbued these artifacts with a kind of transcendent power. Rather than focusing on how to use science and technology to improve the human condition, we had come to believe that our instruments could connect us to higher truths.

– p.239, God, Human, Animal, Machine (O’Gieblyn, Meghan)

Empiricism and Science

Empiricism is the bedrock of modern science. Empiricism is the claim that “more data will eventually lead to more knowledge.” We can see that more data did not lead to more knowledge; if anything, more data has lead to more buying and selling of goods and services. Visualizing the data that almost all technological platforms collect itself requires a bevy of software. Organizations that collect user data are chaotic enough that companies claiming to help organizations “get their data in control” have increased rapidly. We frequently talk about data warehouses now. The problem is no longer the lack of data; it is the abundance of it. (There is a truism everyone is tried of hearing.)

So, a lot of data did not necessarily lead to more knowledge. How about the scientific process? Was that improved at least? No. Here, O’Gieblyn points out the most basic of insights: The whole point of Science was to understand how things worked, how mechanisms could be explained; in a sentence, it was to understand how an input (coal) could lead to an output (power). The more data we had about the input, the output, and the process that happened in between, the more we could improve the machine. Steam engines are far more efficient now than when they were first invented. A lot of that was down to more data collection about the various factors which affect efficiency, mathematical formulations about what leads to more efficiency, and experimental proofs that proved the formulations right. This cycle is no longer in place anymore.

Every company that dabbles in the “product” business has a “data science” team. Using the most basic statistical insights (namely, the test for statistical significance), these teams run A/B tests: assigning users randomly to two different experiences and measuring which one makes them more money. While the “sober” attitude towards testing is that there is a lot of “user research” which tells the company what users want, and then the company runs an A/B test to confirm that what users want is really what makes the company the most amount of money, the real attitude is quite succinctly summarized in a couple sentences: “Who knows why people do what they do. They do it and we can measure that with very high fidelity!”

While normal A/B tests are still run by actual engineers and analysts who find insights in some data, machine learning systems automate this process by setting a mathematical goal in the beginning and letting the algorithm choose among several variants and find the best variant in a self-feeding feedback loop. This is the nature of most recommendation algorithms that are used to decide which TV show should be at the top of a on-demand movie streaming service, and which thumbnail should be used for the TV show, with the goal to make the user click on the show and watch a few minutes of it. This problem is exacerbated in opaque industries like programmatic advertising where decisions about which ad should be shown must be made within a fraction of a second.

Book of Job

The conclusions of this recommendation “algorithm” are to be accepted without questions about or attempts to understand how they were reached. This approach is commonly called science, but as O’Gieblyn neatly points out, is indistinguishable from the concept of an omniscient God. At the beginning of the Protestant revolution, Calvin and Luther were priests who said that God was radically other than human and it was futile to try to understand why God did what He did. It was man’s duty to simply accept what was written in scripture and receive all knowledge as revelation.

The rise of ML explainability is a sign of the apparent inscrutability of systems which are being built. However, the system which is used to explain the original system is itself independent of it and a completely different algorithm which is only building a “narrative” about the decision, rather than actually “looking inside.” Their inherent nature of being a faceless algorithm gives them a veneer of neutrality and strong protection against any claims that they are unjust in some humanly understandable way. The problem starts even before the system starts making decisions (“inferences”):

Moreover, because the training data sets are so large, it’s hard to audit them to check for these embedded biases. “A methodology that relies on datasets too large to document is therefore inherently risky,” the researchers conclude. “While documentation allows for potential accountability, […] undocumented training data perpetuates harm without recourse.”

– We read the paper that forced Timnit Gebru out of Google. Here’s what it says. | MIT Technology Review (Retrieved: [2023-10-09 Mon 16:09])

When the system finally starts making predictions, its predictions can change behavior. Systems such as Predictive Policing (PredPol) were designed to anticipate the neighborhoods with the most crime; instead, they ended up reinforcing their sample data and producing the very outcome they predicted because when the police went to a locality hoping to find crime, they often found it.

Before the Cartesian Partition

At long last, one is convinced that Descartes’ thought experiment, which lead to the mind-body partition, gave rise to modern science and modern hype. What came before it though? Here again O’Gieblyn does not disappoint. She goes into the history of ideas before the partition, through the lens of other philosophers.

Plato’s philosophical system gave primacy to ideas and forms. The World of Forms was always present and never changing; it was only in that world that the perfect form of an object existed. There was a separate World of Matter where these objects existed, and they had a part of the characteristics of the perfect form but they could never hope to be completely identical to the perfect form which was unattainable. So, in Plato’s thinking, these “Universals” were characteristics which were always present, always certain, and most importantly, “common” or “generic.” These universals did not need to be proved.

The Loss of Universals came about in the Late Middle Ages (late 14th century), when suggestions began that there was nothing universal or common about ideas at all, and that it was just the names that we give to commonalities we perceive in objects. This was the Nominalist style of thinking. Nominalism eventually found its way into the mainstream through Protestantism and subsequently spawned the disenchantment and the scientific revolution.

Blumenberg’s thesis, which has since been reiterated by a number of philosophers and historians, is that nominalism, as it became widespread in Protestant theology, led to the Enlightenment, disenchantment, and the scientific revolution. The trauma of lost universals created an intolerable situation, one that reached the point of crisis in the thought experiments of Descartes, who so mistrusted his own powers of reason that it was not inconceivable, he imagined, that God somehow deceived him into thinking that a square had four sides or that two plus three equaled five.

– p.215

This is another path down which we find our work-obsessed culture. Protestant theology encouraged a work ethic which was powered by the anxiety of not knowing one’s predestination. (The concept of predestination decreed that one’s destination after death, hell or heaven, has been decided and will remain unaffected by one’s conduct during life.) Predestination, coupled with the Protestant concept that work benefits the individual and society as a whole, transformed the nature of jobs themselves, making it an obligation to consistently work diligently.¹ These conditions were perfect for the rise of Industrial Capitalism, in which the harder people work for the same wage, the cheaper the products that are produced, the lower their wages will soon be, and counter-intuitively the harder they must work to earn the same wage as they did before.

Protestant work ethic - Wikipedia ↩

Review - Subprime Attention Crisis (Hwang)

2023-07-23T00:00:00+00:00

Rating: 4/5

What would happen if all the free services that you use on the Internet, which are powered by advertisements, stop being free one day? This is the premise of the Subprime Attention Crisis. If you ask yourself this, you might realize that many of the services that we think of as free are powered by advertising. For me, the most frequently used services that are ostensible free are Google Maps, WhatsApp and YouTube. I use at least one of them almost every day. If they were to become paid services, or the hurdles to using them without giving up too much data increased, it would be mildly annoying. This calculus will be very different for an Internet user who uses Gmail and uses it to receive important communication. Hwang’s argument is sensible and easy to understand. He starts from the basics of advertising on the Internet and builds up to his myriad theses: Programmatic advertising is very similar to the financial markets. Ad networks claims that targeted advertising on the Internet is better than “spray everyone” advertising on TV. This claim is a lie and that banner ads don’t really change consumer behavior. Commercial interruptions are blocked by users using ad blocking plugins or because users reliably skip ads on video-only platforms in under half a second. To back all of this, he presents a lot of industry research and anecdotal evidence. This was a convincing case for being aware that free services could stop being free any day, and there would be nothing really surprising about it. (Just the other day, YouTube took a step towards blocking ad blockers.)

Concrete examples are helpful to set the stage:

Gmail is a free, e-mail service that launched in 2004. In the past 19 years, it has grown rapidly and now has 1.5 billion active users. Gmail.com has become the most common e-mail domain. Gmail is run by Google, a search engine company, which had a total revenue of $70 billion between January and March 2023. 78% of this revenue came from Google’s advertising network.¹
Facebook (now Meta) is the company behind 3 of the most common messaging and social media apps today: Instagram, Messenger, and WhatsApp. Facebook was launched in 2004, WhatsApp in 2009 and Instagram in 2012. Both Facebook and Instagram are powered by ad revenue, while WhatsApp is a popular end-to-end encrypted, ad free messaging platform. 3 billion people used at least one of these apps in March 2023. Facebook’s total revenue between January and March 2023 was $28.6 billion. 98% of this revenue came from advertising.²

What is Programmatic Advertising?

Programmatic advertising is a marketplace with buyers and sellers. Publishers (like the New York Times news website) sell space on their websites, where ads can be displayed. Buyers (like a local business) buy this space to display an advertisement about the goods or services that they provide. The commodity that is being traded (in the abstract) is the attention of the person viewing the publisher’s web page. The process that decides which ad should show up on a webpage when a user is visiting it is called Real-Time Bidding (RTB). RTB is a sub-100ms auction where the buyers’ ad networks look at the data that is available about the user and decide how much they are willing to pay to show a given ad to them. This process is easy to imagine when a few transactions are happening in an hour. But the process happens at a much larger scale and is completely automated. The technology behind it is probably complex, and (as Hwang demonstrates) publishers often don’t even know which ad network served a given ad on their website.³

Standardizing the Marketplace

There is one hiccup to setting up this marketplace. When exactly can we say that an ad has been shown to the user? In other words, when exactly has the user’s attention been captured? This is clearly hard to define. In a long news article, if the ad was displayed after the 10th paragraph but the user stopped reading the article or clicked on another link after looking at the headline, the ad was not viewed and the publisher should not be paid for placing an ad there. These things are specific and explain what ads should look like and when they should be considered to be “delivered.” I looked through the website of the Interactive Advertising Bureau (IAB), which sets these standards to find 2 interesting examples:

For ads that are delivered to a desktop web browser, ad measurement happens when the ad content has been downloaded by the client and counted as delivered when it begins to render. The actual guideline goes into excruciating detail and tries to define everything:

An Ad Impression across all display marketing channels is the measurement of responses from an ad delivery system to an ad request from the user’s browser, which is filtered for invalid traffic and is recorded at a point as late as possible in the process of delivery of the creative material to the user’s browser. The ad must be loaded and at minimum begin to render in order to count it as a valid ad impression. Measurement of begin to render should include logical components necessary to display the ad, but does not necessarily include logical elements that are not essential (such as other tracking elements)

– https://www.iab.com/wp-content/uploads/2017/11/Desktop-Display-Impression-Measurement-Guidelines-US-MMTF-Final-v1.1.pdf (p.5)

For podcasts (an audio medium), there are two metrics. The first one measures whether the file containing the ad was delivered:

Ad Delivered: an ad that was delivered as determined by server logs that show either all bytes of the ad file were sent or the bytes representing the portion of the podcast file containing the ad file was downloaded.

– https://iabtechlab.com/wp-content/uploads/2021/03/PodcastMeasurement_v2.1.pdf (p.16)

While the second one measures whether the podcast client can confirm that the ad was played:

Client-Confirmed Ad Play: counts an ad that was able to prompt a tracking beacon from the client when the file was played. Whenever possible, metric should include information about how much of the ad was played using the markers: ad start, first quartile (25%), midpoint (50%), third quartile (75%), and complete (100%).

– https://iabtechlab.com/wp-content/uploads/2021/03/PodcastMeasurement_v2.1.pdf (p.16)

Like commodities trading, the attention that is being traded in these IAB standards is different from the underlying “thing.” It has been standardized and defined in an effort to attain an unattainable perfect commodification of the user’s attention.

How is Programmatic Advertising Different?

Advertising is not a new concept. Newspapers have been funded by classified sections and outright advertisements for at least a century. Hwang identifies 3 things that are different about programmatic advertising:

The speed at which advertisers decide whether or not to serve an ad to a user on a webpage is much faster than before. Before buying an ad on a television show, or a newspaper, ad buyers would generally do months of research about viewership/readership data. This data was not perfect either, but the time that these decisions took was much longer.
The volume of ads that are served to users per minute has increased. On TV, a 30 minute program has 22 minutes of content and 8 minutes of advertisements. Online, a single news article which one would read in about 5 minutes can have 2 banner advertisments. Video streaming platforms are worse with 30 second advertisements for videos which are less than 5 minutes long in some cases. There is no way to say how much exactly the volume has increased. But the lack of this data should not prevent us from noticing that the number of ads we see on a daily basis has skyrocketed.
The granularity with which ads can be placed within a piece of content has increased. A newspaper has a given section for Classifieds, magazines assign pages or the back cover for advertisements, and TV shows demarcate content and advertisements very clearly. (“Mute the commercials, will you?”) Online, where ads are placed can be controlled right down to the paragraphs between which the ad shows up or the section of a video in which the ad would be most effective.

Similarities to Finance

The executives who were building advertising on the early Internet and trying to figure out how to make these new services profitable were from the finance world. For e.g., Sheryl Sandberg, widely regarded as the person who was responsible for teaching small businesses to use programmatic advertising, had a background at the World Bank and the United States Treasury. The model that they chose to replicate for programmatic advertising was the one of high-frequency trading.

The most important similarity is market opacity. When the commodity that is being traded is not well understood or its value is misunderstood by market participants, a crisis of confidence is always looming around the corner. This is what Raghuram Rajan and Reinhart have both argued are the prerequisites for the creation of a bubble in any economy. The crisis of confidence will trigger the boom/bust cycle as in the financial markets, as marketers realize that the money they spend on programmatic advertising is not delivering the returns that they were promised and stop spending as much. One question here is whether the old markets of advertising (television, radio) were not just as opaque? Of course, they were opaque too. However, the marketing that was done on TV or radio did not claim to be targeted or very good at anything except raising brand awareness in the first place. A famous ad industry adage is “Half of the money I spend advertising my product is wasted. But I don’t know which half.” The opacity in the market was accepted as a condition. Programmatic advertising makes unattainable claims about being targeted and measurable; while simultaneously, publishers are not sure which ad network is delivering what ads to their users.

Another similarity that Hwang talks about is the “private market.” Companies like Facebook and Google have premium ad space which is sold on private markets. This limits the information available in the public market, and makes it hard for advertisers to figure out how much a given webpage is really worth paying for.

A Bubble in The Attention Economy

First, ads don’t work. When WIRED, one of the early websites, ran a banner ad, 40% of people who saw it clicked the ad. Today, even 1% is a pipe dream, with the average being about 0.2-0.4% of viewers engaging with ads. One way to evade this critique is by saying that advertisers are doing “brand advertising.” Brand advertising simply focuses on showing people brand names and products and does not aim for any change in behavior at all. This allows advertisers to make the duplicitous claim that programmatic advertising works at increasing brand awareness, without measuring anything or changing anybody’s behavior.

Second, ad blocking is on the rise. Google admitted in 2014 that 56% of ads that were displayed are never seen by a human. 800 million devices had ad blocking enabled in 2022. The demographic trend is also discouraging for advertisers: Younger users and wealthier users use ad blockers at a disproportionately high rate. In this section, Hwang reveals another stunning fact that makes total sense once you read it: Snapchat’s users skip commercial interruptions within less than a second. Less than a second.⁴ Snapchat’s users skew younger and are clearly very ad-averse.

Third, click farms inflate the value of advertisement in some domains. There are also the more technical problems of domain spoofing, where unreliable and obscure ad networks sell an add on the domain NYTemis.com, but brand it as one that is going to appear on NYTimes.com. Buyers and publishers are not able to detect this behavior, and the technical solutions which prevent such domain spoofing when visiting websites is still not available when buying ad space.

Fourth, middlemen. There are resellers and middlemen who add a markup for no value at all. The marketing agencies which used to be premium places for creating ads have lost ground in that area, but they are active in this market of procuring discounted ad space from the ad networks, adding a mark up, and reselling the space to some buyer. These middlemen get a commission on the ad space that is bought and are completely disconnected from any effect of whether the ad actually changes consumer behavior. (Does this sound vaguely familiar? In the movie The Big Short (2015), about the 2008 financial crisis, these middlemen are the mortgage brokers in Florida who were selling condos by giving subprime loans to anyone with a pulse, and pocketing the commissions on these loans.⁵ The middlemen brag about the loans as “NINJA loans: No Income No Job” and talk about how the bonuses on these loans started skyrocketing a few years ago. In fact, Steve Carrel’s character asks them the very question that Hwang complains that no one is asking in the programmatic advertising business: “Do people have any idea what they are buying?”)

The way that Hwang writes, the similarities between the financial markets and the advertising market, and the time before the crisis in the financial market and now in the advertising market, become quite clear. The whole book is obsessed with bringing these similarities out and showing what is going on.

What Next?

What is an advertisement supposed to do? Ignoring the amorphous category of “brand advertising,” all of advertising has the same goal: Change the viewer’s behavior in a predefined, intended way. The online attention economy is based on the confidence that ads on the Internet, bought through ad networks and sold through real-time bidding, are more effective than old-school, “spray everybody” advertising. A crisis of confidence will happen when buyers stop advertising through this method and realize that it does not affect their bottom line adversely at all. Buyers and sellers have already started doing this⁶ and they are not seeing their revenues drop.

The ways that we use the Internet are constrained to the mediums in which advertising can spread: text with engagement signals like “Like” and “Favorite”; quantifiable popularity metrics like follower and subscriber counts. A place that enables computers to connect to each other without commodifying everything that people do in that place does not necessarily have to look like our current Internet. The way that the Internet is funded is important to the way that the Internet is. The algorithms put inflammatory posts that make people mad at the top of their feeds because that is what sells advertisements and motivates people to take action. Hwang’s optimism is touching here: He says that the Internet had other routes; it could have relied on other sources of revenue. I believe him. People could have run their own servers if the technology had been simple or if the education ministries had kept up with the technology. Long-form blogs would have been the order of the day, rather than “outrage of the hour” sentences. People would have found it harder to target others because interaction was curated and disperse.

We should care about how the programmatic advertising market works because it is the main source of funding for many free services today. This article links to a few YouTube videos and news articles, which remain free because YouTube and publishers have ads on their website, which they believe is making them enough money that they don’t want to stop people from just sharing links around. If that wasn’t the case, how would you share video clips with everyone? Hosting is not free, and neither is bandwidth. The collapse of this market would change the Internet completely. And possibly, for the better.

I keep returning to Hwang’s premise for this book. What would happen if all the free services that you use on the Internet, most of which are powered by advertisements, stop being free one day?

Google’s financial results for January-March 2023 (PDF, p.2). Total revenue: $69,787 million. Revenue from Google advertising was $54,661 million. ↩
Facebook’s financial results for January-March 2023 (PDF, p.10). Total revenue: $28,645 million. Revenue from advertising was $28,101 million. ↩
This was quite clear during the YouTube Adpocalypse scandal when YouTube was struggling to control ads that were being displayed next to content that was affecting “Brand safety,” the concept of surfacing brand names next to agreeable content. ↩
I could not really believe this fact, so I dug up the article where this fact was sourced from: “Advertisers and media partners familiar with the company’s thinking say serious consideration is being given to such a plan, which would help solve a nagging flaw in Snapchat’s business: its young users often skip commercial interruptions within less than a second.” (Snapchat May Force Users to Watch Three Seconds of Ads Before Skipping - AdAge) ↩
This is the clip from The Big Short (2015). ↩
Hwang mentions the New York Times moving away from programmatic advertising, and other examples in the book, but I have not put references to those here. I found 2 interesting examples for ad buyers and sellers: First, JP Morgan Chase, the consumer banking devision of the large investment bank JP Morgan: “Of the 400,000 web addresses JPMorgan’s ads showed up on in a recent 30-day period, said Ms. Lemkau, only 12,000, or 3 percent, led to activity beyond an impression. An intern then manually clicked on each of those addresses to ensure that the websites were ones the company wanted to advertise on. About 7,000 of them were not, winnowing the group to 5,000. The shift has been easier to execute than expected, Ms. Lemkau said, even as some in the industry warned the company that it risked missing out on audience ‘reach’ and efficiency.” from Chase Had Ads on 400,000 Sites. Then on Just 5,000. Same Results. Second, Bloomberg, a major publisher of financial news, stopped open-market programmatic advertising. They moved to the time tested strategy of asking buyers to work directly with their media team to place ads. The stated reason for this was the performance of their website. Notably, Bloomberg shut down their ad sales despite seeing the revenues from this business grow for more than 2 consecutive years. “Starting January 1, 2023, Bloomberg Media will no longer allow third parties to sell ads to our audience through open-market third-party programmatic, or other non-direct sold “demand channels,” across our website and apps. Going forward, if brands want to reach our audience, they’ll need to work directly with our world class media team.” – Shifting to an Audience-First Mentality - Bloomberg Media ↩

New Wireguard Peer in 10 Minutes

2023-07-22T00:00:00+00:00

I have been using Wireguard for about 4 years now. I started with the basic Wireguard setup, where I did everything manually and used the ip link and ip addr commands to create new links and assign addresses to them. The wg-quick utility is open-source and has built-in SystemD support. So, I switched to using the configuration file that wg-quick requires. With an ad-blocking DNS server, I was able to use the setup for a long time without having to change anything at all. The client support was excellent on Linux, Android and (even) iOS. I never really noticed whether I had my VPN enabled. I kept it enabled at all times except when I needed to access some geofenced service. However, I was still writing the wg-quick configuration files manually and most of them were very similar. This caused me to put off adding new peers to my network as soon as I needed to. Recently, I wrote a Golang CLI tool which generates wg-quick compatible configurations for Wireguard peers, based on a simple JSON input file. Using this tool, I was able to add a new Wireguard peer and create a new network in less than 10 minutes. The path to that CLI tool is what this post is about.

Wireguard is very simple.¹ Literally and conceptually. Literally, it is less than 4000 lines of code. The original patch that added the VPN to the Linux kernel is very small and I have always wanted to read it, although I have never been able to because knowledge about the Linux kernel networking stack is a prerequisite. Conceptually, it can be explained with a single slide. A rudimentary understanding of Linux syscalls is sufficient to understand this slide.

Source: Wireguard Presentation - October 21, 2016 - Codeblue, Tokyo, Japan

Given all of this simplicity, the wg-quick configuration to establish a traditional VPN is also very simple: It is about 10 lines of code that needs to be on every machine that is part of the VPN, and the 2 main differences between configuration files were the ASCII keypairs that are used to identify peers inside a network, and whether it is on the server (the peer which will forward all the traffic towards the Internet) or the client (the set of peers which will send traffic to the server asking it to forward it to the Internet). The configuration is so small that no tool is distributed to generate it.

I have 2 main devices: a computer and a phone. I was able to connect them to 2 VPN servers with a total of 6 configuration files. The key pairs for each pair among the 3 servers for 2 different networks was different (intentionally); so when I was originally setting this up, I made some mistakes while copying the keys and pasting them around, and it took me a couple tries to get the setup right. Once I got the setup right, I did not bother to change it for 2 years. After that, I had some new requirements which this manual setup did not meet: I wanted to add new clients and new servers quickly, because I expected the number of clients to grow and I did not want to manage the keys manually anymore. As each peer had a single private key inside each network, and the public key could be derived from the private key, this only meant that I needed some sort of persistent map between 2 keys: network and client name and 1 string value: the Wireguard private key. JSON or YAML files are perfect for this and building a system which reads the file and writes configuration files to disk would be ideal and easy.

Some more advantages of this setup was that, all of these configurations were just files. There is no need to introduce a new tool to manage them. If your hard drive is already encrypted, you can just keep the files as-is on disk, and as long as you are able to boot your computer, the file will always be available. If you want to keep the file on some sort of cloud storage but encrypt it end-to-end, then you could easily use GNU Tomb to create an image file, that is internally a LUKS-container, with an ext4 filesystem, and upload this image file to the cloud storage system. Most cloud storage systems have good integration with the Gnome Files or Nautilus applications, so syncing the files is not manual and the status of syncing can be checked easily.

I did not get around to implementing all of this until last month, when I realized that one of my new devices did not have a VPN setup and I had been avoiding any use of the device that required a VPN. I did not want this pattern to continue. I wanted to be able to introduce new clients and servers easily and without having to think too much, so I got around to writing a single-file Go program which would use a JSON file as input and generate configuration files on demand.

The result of it is icyflame/wireguard-configuration-generator. It is not a single file application anymore, and I ended up adding some convenience features. My favorite one is the input file validator. The JSON file itself is quite small and you can probably read the whole thing in one glance, however, when adding a new peer, you have to ensure that the IP address that you assign it is unique. So, the tool will automatically read the whole file and ensure that it makes sense: That each peer inside a given network has a unique IP address and that the names of the peers inside a network are unique so that the files can be properly named.

There are many great talks about Wireguard by the author, zx2c4. I recommend this one: WireGuard: Next Generation Secure Network Tunnel - YouTube ↩

Conversion from HTML to ePub Format

2023-06-17T00:00:00+00:00

Recently, I have taken to reading magazine articles and long newsletter posts on my Kindle by converting them to the ePub format, rather than reading them on the computer, where the process of making highlights and taking notes differs from the process that I use for all the e-books that I read. As I started doing this for some long articles (such as this one), I realized that the best online options out there are not good enough. I have been using dotepub.com which seems popular and converts to both the generic Epub format, and the Kindle-specific Mobi format. While it does a good job with all the text, this particular article was particularly heavy on images, and all the images were required to understand the text. When I converted the page to an Epub format, it told me that it would not include all the images from the article. So, I set out to write a few scripts which could fix that problem and actually export web pages as self-contained epub files.

Aside: You can skip this explanation and directly read the scripts here.

Dotepub’s hesitation to download all the images, and then include them in the epub file, is probably an attempt to restrict the amount of bandwidth that is consumed by each user. As a user-facing service, where the developer would have to spend quite some time to prevent abuse, I understand that. While the epub file will certainly get bigger when you include all the images from an article, I doubt that anyone on a broadband connection will mind that. And they have anyway loaded all the images locally on their computer once. Downloading an Epub container with the same images might be something that they explicitly want to do (and that’s why they are using dotepub in the first place.)

Before converting the raw HTML of the webpage into an Epub file though, it is important to understand the structure of the Epub file itself. And to do this, I checked out this talk from the BlackHat Europe 2021 conference (Talk video). The talk has a clear explanation of what an Epub file is (just a set of HTML, JS, CSS. Sound familiar?), what are its common constituents (some manifests files that point to various locations with the data, the images), and the engines that ePub readers use to render the content of ePub files. After this, I looked at some of the real world ePub files that one can download from Project Gutenberg to see how these are implemented in practice. I was pleasantly surprised to see that the XML manifest itself was short, easy to understand, and well-documented; and that it had very few required fields; with most of the fields being optional. I guess eBook readers have to work with a huge variety of files so they impose the least amount of constraints possible.

Around this time, I also found out about the Readability library, the library that is used by Firefox to generate their excellent “Reader view” for pages with a lot of text. I wanted to use this library because it would give me a cleaner, more ebook friendly version of the text that is generally on webpages. Most magazine websites have sidebars, headers, and footers with links to other articles or sections. All of these would just be distracting on an ereader device, which is designed to take a blob of text and render it in a pleasant way.

I also found the DOMPurify library, which sanitizes HTML and removes any cross-site scripting (XSS) vulnerabilities from HTML. This is important because, as mentioned in the Blackhat talk, e-readers do have the ability to run JavaScript. Even though my Kindle is not connected to a network most of the time, it is still possible that the JavaScript that is part of these HTML pages reaches out to external services (like Analytics.) I don’t want these external requests to be made from my Kindle; an environment which does not have the convenience of installing the uBlock Origin extension.¹

I experimented with some methods of downloading the webpage. I tried to download the HTML, and other associated resources (CSS, images) manually, using some Node.js/Go code. I gave up on this approach pretty quickly, because I realized that I was just re-implementing what every browser’s rendering engine already does well. Then, I used the “Save” feature built into Firefox. This feature is very good and works well when you want to save just the images on a page. But it did not work well to create an exact copy of the page, because many of the resources that were referred to on the webpage could be served from another domain (such as a webpage example.com/article which downloads static images, CSS and JavaScript from static.example.com.) This is a common pattern and followed by many websites. So, consider that example.com/article refers to 2 images example.com/images/first.png and static.example.com/images/second.png. On saving the webpage, the first path would be changed to /images/first.png and rendered properly. However, the second path would be left untouched by Firefox. So, the simple Save feature did not work either.

My third attempt worked well: SingleFileZ. SingleFileZ is a browser extension which allows you to download all the resources that are required to render a webpage as a zip archive which contains an HTML file and its associated resources. The paths to the resources are also appropriately changed. As SingleFileZ attempts to create a copy of the webpage that will work without any network connection, it downloads everything, renames those files, and prepares a new HTML file which is completely self-sufficient. (It simply does not care about which domain the resource originally came from.) Also, files are neatly arranged based on their type; i.e. images are all put inside a single folder. This extension is really useful even if you plan to just re-read magazine articles directly on a computer.

After downloading the HTML, and putting it through DOMPurify and Readability, I planned to use Pandoc (inside a Docker container) to convert the resulting HTML file into an Epub file.

With these three pieces, the only part left was to write some Node.js code to deal with the HTML and some Bash script which would invoke these tools in the correct order. This was fairly simple. One of the hiccups that I ran into along the way was that the HTML that is produced by many websites is not valid (i.e. some closing tags are omitted). Pandoc expects strict HTML as input. So, I had to convert from HTML to XHTML using Pandoc once, and then convert from XHTML to Epub in the second step. This worked well and produced very readable Epub files.

There are still many shortcomings in the process. The most glaring one is that there is no cover image for the Epub files that are generated. Although we have all the images from the webpage, deciding which one should be the cover image would require manual intervention. For now, I use the Kindle’s Library List view which shows the title of the books as a list, rather than the (visually pleasing) Covers view, which shows a bookshelf with book covers arranged on it.

Scripts on GitHub

I know why this extension is not a default on all browsers: because it will hurt the bottom lines of the companies making the browsers, which also happen to be running, by complete coincidence, the biggest advertisements networks. But I don’t know why users are still using the Internet on a browser which does not have this extension installed. Install uBlock Origin now! ↩

Review - Darkness at Noon (Koestler)

2023-06-04T00:00:00+00:00

Rating: 5/5

This novel starts off with an imprisoned protagonist thinking of the past where the prisoner was part of the very Revolution that has now imprisoned them. As the story progresses, philosophical ramblings come at increasingly frequent intervals and the novel reveals the most valuable plot point: watching the protagonist go back and forth between the belief that the “Revolution” was a good thing, which will eventually attain its original goals; and the belief that the Revolution was incorrect to say that “an individual is the product of one million divided by one million.”

Other dystopia tell us about the villain well before the book starts: No one thinks what Big Brother does in 1984 is doing could be remotely good. No one thinks that burning books in Fahrenheit 451 is the logical thing. The outcome is framed, forcefully and without any shred of doubt, as an aberration; a mistake. The author is trying to get the reader to realize the absurdity of the situation and the mistakes in reasoning that lead to it. While reading Darkness at Noon though, this is not always clear. I am fairly convinced that the Revolution did not actually change anything in this fictional world and that had it not happened nothing significant would have been lost. However, we get to see the present through Rubashov’s eyes, and Rubashov himself was deeply involved in creating the Revolution and putting No. 1 in power. So, when Rubashov explains the reason for the way things were done during the Revolution, it is clear (to us and to Rubashov himself) why Rubashov himself has now been imprisoned by No. 1.

The story starts off with the protagonist of this novel, Rubashov, switching relentlessly between the present and the past. It took me some time to understand what was going on, whose story this is, where we were. I managed to piece together most of this by p.70. But then, from p.73 to p.76, Koestler has another character read Rubashov’s full biography. Nevertheless, this part of the book was fun. So, one thing I will say is this: *Do not obsess over the specifics of where the revolution is happening, what Rubashov’s exact role was, how No. 1 came to power, what happened to the old leader, etc.* These are mostly peripheral to the main question of the plot: *Should Rubashov participate in his own self-sacrifice for the sake of the Revolution? Or should he “die in silence” and betray the Revolutionary cause, which has become increasingly unclear to him?* Just as Rubashov was instrumental in the death of a union member who believed the Party’s ideals too much, now No. 1 has come to him and asked for his head. Should Rubashov acquiesce?

During the course of the hearings, we see that Rubashov thinks everything out to the logical conclusion: He concludes that “the Party can never be mistaken.” He decides to confess to the charges that are laid upon him, under considerable duress, and agrees to go to a public trial. The tactics used on him are (ironically) the same that worked on common peasants: sleep deprivation, constant interrogation with brief periods of rest of a couple hours at most, a fluid set of charges that are applied on him and updated on the fly depending on what actually confesses to.

But he also ignores some key flaws in his thinking: He says that he is acting honorably, and that to act with honor is to “be useful without vanity.” From the descriptions of what No. 1 did to his opponents or is now doing to Rubashov himself, it is clear that No. 1’s primary characteristic is his vanity. There is no honor in what No. 1 is doing. He has lied to his population that Rubashov is a traitor to the Revolution; and he has lied to Rubashov (indirectly) that the population needs a traitor in order to be invigorated about the Revolution; that there will be a war soon and to give them a public trial is the only way to reinforce their true belief (or at least, weed out more “traitors”). No one benefits except No. 1, who stays in power perennially, through the application of this devious logic.

One final thing is about revolutions themselves. The book only lightly touches on this point, but it felt very insightful to me. Rubashov was a part of the original Revolution; he was fighting against economic fatality; he was applying philosophy to problems in order to get the population to revolt against Capitalism. The generation after the Revolution’s founders are also ostensibly doing the same thing; however, there is one key difference: They have never seen the world before the Revolution. In essence, they have no idea what they are rebelling against. They were born when the world had already been transformed violently; and now, they knew nothing about the transformation or how the world was before the transformation. This breaks the whole setup: The Revolution is reduced to nothing more than a “mythical event” which is used to keep things going the same way as they currently are; to retain power within the same centers and the same people. There is no possibility for any true “Revolution” which remains both true to its original goals and in constant flux. For the original goals will soon be forgotten. And the flux that overthrew the old ruling class to put the new one in place, if allowed to continue, might be used to throw out the new ruling class and put an even newer one in its place. This is the ultimate logical wrench in the wheel.

Rubashov is unable to find any way to pull the wrench out. But he is hopeful:

[Rubashov thought] Soon it would be over. But when he asked himself, For what actually are you dying? he found no answer. It was a mistake in the system; perhaps it lay in the precept which until now he had held to be incontestable, in whose name he had sacrificed others and was himself being sacrificed: in the precept, that the end justifies the means. It was this sentence which had killed the great fraternity of the Revolution and made them all run amuck. What had he once written in his diary? ‘We have thrown overboard all conventions, our sole guiding principle is that of consequent logica; we are sailing without ethical ballast.’ Perhaps the heart of the evil lay there. Perhaps it did not suit mankind to sail without ballast. And perhaps reason alone was a defective compass, which led one on such a winding, twisted course that the goal finally disappeared in the mist. Perhaps now would come the time of great darkness. Perhaps later, much later, the new movement would arise – with new flags, a new spirit knowing of both: of economic fatality and the ‘oceanic sense’. Perhaps the members of the new party will wear monks’ cowls, and preach that only purity of means can justify the ends. Perhaps they will teach that the tenet is wrong which says that a man is the product of one million divided by one million, and will introduce a new kind of artithmetic based on multiplication: on the joining of a million individuals to form a new entity which no longer an amorphous mass, will develop a consciousness and an individuality of its own, with an ‘oceanic feeling’ increased a million fold, in unlimited yet self-contained space.

– p.206

Better Personal Data Management

2023-04-08T00:00:00+00:00

I switched from Ubuntu 22.04 to Debian 11 earlier this year. I was using Ubuntu’s LTS OS versions for nearly 9 years before that, since 2014. The switch to Debian was because I stopped liking what Ubuntu had done with the OS: The GUI has become a strange mix of Ubuntu’s old desktop environment (Unity) and the Gnome Desktop Environment, which is standard and popular. I generally use the i3 window manager, so the Desktop environment was not too annoying; I could have lived with it if not for Snaps. The introduction and use of Snaps was a continuous thorn in my setup. Each snap sets up a new loop file.¹ So, the output of df is polluted with these strange loop devices that I don’t care about. Also, Firefox installed using Snaps does not work well with the KeepassXC Browser integration. So, the recommended way for installing programs on Ubuntu was actively getting in my way. (There is also some Ubuntu bloatware but an equivalent of that is there in almost every other non-base distribution, so it can’t really be counted against any single distribution.) I wanted to do a few things differently with this reinstall.

Frequent OS Version Updates

Over 9 years of using Ubuntu, I used only 3 LTS versions. I was hoping that I would not have to go through the OS installation process multiple times, which would save me a bunch of time to learn and set it up properly each time. But this approach was wrong. Every time that I thought I would not have to install the OS again for at least 5 years, my plans were thwarted and I had to install the OS within 3 years or less. This was due to first, a new laptop, then a crashed HDD, then a new laptop, then a glass of water spilled on my (spill-proof) laptop keyboard, and finally, another new laptop (a cheap replacement for the one that I ruined.).² So, now my target is to update to the latest version of the OS every year. Of course, it comes with recent Linux Kernel versions, but also ensures that I am on the latest version of most packages. (I had a lot of problems running Ubuntu 16.04 LTS on my old laptop, barely 1 year after its end of life, because a lot of the packages that I needed were not being packaged for that version of the OS at all. So, I had to manually build a lot of things. And each of those things had its own dependencies, which were also not packaged for older versions of the OS, … and so on.) Also, the increased speed that comes from running the latest software on a very old device is stunning. After upgrading from Ubuntu 16.04 (released 2016) to Ubuntu 22.04 (released 2022) on a 9 year old i5 Dell Inspiron machine, the computer became usable. It was totally fine for the normal mix of writing / reading / Netflix / YouTube. The laptop has a hard drive, which I suspect caused the fans to spin up when I was streaming video, but everything else worked well.

Decouple Data from the OS

Regrettably, until a few days ago, I was blasé about managing my data properly. There were a bunch of folders, with a lot of files, and they were all sort of required, but I never really knew which external hard drive to check for what files. Backups were not regular or complete. I used Dropbox for some use-cases, but not for others (which doesn’t make much sense.) It isn’t like this approach had not already caused me problems: I have accidentally deleted and (through a painful process) recovered a folder containing all my files from college twice, and both times because I was trying to get all this data under control, but I was doing it with the incorrect rsync command. On my computer, I was used to putting all my files inside the home directory. There were a few files scattered around in ~/code and ~/personal. It was really an unmoderated (and embarrassing) shit show. Linux provides the elegant concept of partitions. The /boot partition consists of your kernel and things that should be kept in plaintext, the / (root) partition consists your OS files and /etc, /var etc: These should be the things required to login. Finally, there is the /home partition which consists of things that you would use only after logging in: SSH and GPG keys, configurations for your software, your shell configuration, personal files, etc. You should be able to reinstall your OS by changing just the /boot and the / (root) partitions. The /home partition can be left untouched by a new OS installation, and should work seamlessly with the next version of the OS. I am going to better implement data management by using various partitions, and ensuring that all of them are encrypted properly.

Encryption

As I mentioned above, every time I thought my computer was basically indestructible (why did I think this?) and that I will not run into a situation where I would have to give it to someone else unlocked, I had to give it someone unlocked. The technicians who service your laptop will undoubtedly ask for the password of a user on the system. This is unavoidable.

The correct way (if there is one?) would be to have a “Support” account with a documented password on all your computers, and keep it free of any personal files and ensure that if you have to hand your PC to someone else and let them connect to WiFi or inspect the PC’s configuration, they can do so without having access to your files. Even this might not be good enough, because some times they might want to do things which require the administrator’s password, and the administrator password often gives them access to your files as well.

The only way to really get around this is to keep the files required to start your computer separate from your personal stuff. Once again, a separate encrypted partition that you manually mount on login seems like the best answer here. The overhead of mounting the partition on each login is tedious, but this can be gotten around by using the TPM or a FIDO device. (Note that if you use your TPM, which is very convenient, then the support technician can technically use the TPM To also mount your devices because the TPM doesn’t check if you are the one who is mounting the device.) Using a FIDO device (YubiKey) is better, but inconvenient because you have to keep the device close to your computer. YubiKey does manufacture low profile FIDO keys which take up one USB port and stay connected to your device, but can be conveniently disconnected before handing the laptop unlocked and over to someone else. That might be a good balance. I am still figuring this out. The trade-off between convenience is security is not a novel debate at all. (The most popular operating system, Windows, gets around it using Bitlocker, which will keep the security key in your TPM and keep a recovery key on Microsoft servers. You can never lose your key and its super convenient, but Microsoft will have a copy of your key.)

Don’t Forget How to Install Operating Systems

This one is controversial and I have had different feelings about it over the past few years. For a while, I loved configuration and tweaking and changing my setup constantly. (The very beginning of my journey with Linux was just installing and uninstalling operating systems on an old Windows desktop machine, crossing my fingers, pressing the power button, and hoping that the computer boots up properly.) Then, for a long time after that (pretty much until the accident with my laptop last year), I craved stability and the same OS version throughout. Now, I am back on the configuration/tweaking part of the cycle, but I don’t want to configure for the sake of configuring. I want to build a configuration which makes it easy to configure in the future.

This is similar to the approach that one would take with an editor like Vim or Emacs: There is a period of intense configuration when you first adopt the editor. But later, things cool down and there is much less to configure and the time spent using the editor far outweighs the time spent configuring it. (One of the efforts with my latest install is that I am using Ansible, a configuration management system that I am familiar with, for doing almost all the installation and building that is required. This is already paying dividends, because I can run this Ansible on a new computer, and get all my software and configuration back. I was surprised at how easy it was to write the initial Ansible configuration, which is the most daunting. The incremental updates to the configuration have been quite easy.)

The thing about installing operating systems on your main computer is that if you don’t know how to do it well enough, you will almost definitely be scared about doing it and keep postponing it. With the limited time that I get, I want to use my computer to do something now; not configure it to do something later. However, there is a point where this logic gets inverted: You keep putting away installing a new operating system. So, you keep forgetting the things you learned to do when you installed an operating system the last time around. Eventually, you end up starting from the beginning and spending way more time installing an operating system than you saved by putting off the installation. I am pretty sure this is the spiral I fell into when I was installing Ubuntu 22.04 after a 4 year hiatus from OS installation on my backup computer last year. The process took nearly 4 days, including all the basic setup, and was quite manual. Compared to that, the Debian installation last month was quick and easy.

Understand the Boot Process

Not knowing what /etc/crypttab and /etc/fstab do, or how to add a new LUKS device, or how to remove a partition from being mounted at start-up, or how to remove the password from one of the LUKS keyslots, etc: These are intricate things that it is not OK to get wrong (because you would not be able to boot properly if you got them wrong.) However, they are also not done frequently enough for you to remember everything. Then, you must rely on your personal notes from the last time you did something (The Zettelkasten system has helped a lot to keep my notes synchronized.) and the learned experience from the last time that you did it. Right now, the past few weeks of configuration updating is fresh in my head. So, I can update the crypttab and fstab file or use the cryptsetup {open,close,luksFormat,luksDump} and the mount / umount commands, without much hesitation. This was not the case when I originally installed Ubuntu 22.04 LTS on my backup laptop or when I installed Debian 11 on my current laptop.

Manage Your Data Yourself

This one might seem like I am fighting the tide of the tide of the “Cloud” here. It is not at all absurd though: You need to build a system, and this takes time and admittedly inconvenient. But once you have it, and start refining it, you can rest easy in the knowledge that your data stays quite close to you and will never cost you more money because the company providing the Cloud decided it was a good time to increase prices.

Take photos for example. They are quite important. When I first set about arranging my photo library early last year, I found that I had taken a lot of photos using various phones that were still stuck on those phones. I had never thought about moving everything over to some central place before, so photos from 2018 were in the phone I used then. Photos from 2019 were in the iPhone I am currently using without any backup. (I don’t use iCloud Photos, so they were stored in just one place.) I know of people who lost a lot of their photos because their phone just refused to start up one day. The helplessness and regret that follows losing important data is neither worth the effort required to build a system to manage data yourselves, nor is it worth saving the money you would pay Google Photos or iCloud, if you are not interested in building your own system. Let’s go back to the start: photos.

Do you take photos with your phone? If yes, do you back up these photos? I have found that there are two answers to this question: The first one is “Yes, I use {iCloud/Google Photos/Dropbox} to backup my photos.” The second one is “No, I don’t backup my photos. Why do I need to? What is going to happen to my phone anyway?” I have found that something always happens.³ When something does happen, but is fixable, you go to the support center. And the first thing that the support person is going to ask you to do is unlock the phone and hand it over to them. This happened both when I gave my Nexus 5X for its motherboard to be replaced and when I went to an Apple Store to get an iPhone’s broken display changed.⁴ There might be the option to not give them your password, and instead ask the support people to just format everything on your phone. You can’t do this if you have photos on the phone that you want to recover. The good faith assumptions about support people not looking at your personal data notwithstanding, do you really want to hand over your unlocked phone with several years worth of personal digital data about you, your friends, and your family to someone else? I don’t.

So, I use Shotwell to store and manage my photo library.⁵ This applies to other parts of your digital life as well: Passwords, scanned data (documents like your passport), email. Taming the data that is created as during my life is worthy of its own blog post. I will probably do at least one about email and photos, because those two have been the most interesting exercise for me. During these, I have run into the strange patterns that keep repeating in managing data, the amazing ability that comes about from thinking of everything as a file, the use of GNU tools like comm, sha256sum, awk, sed, and much more.

So, there it is. My rationale for switching from Ubuntu to Debian, and my rationale for frequently updating your operating system through a controlled process, forcing you to automate as much of the setup as possible. The tools are there and all of them are well-integrated into the Debian and Debian-based distributions.⁶ All you have to do is build your own system.

Loop files are setup using losetup, and allow you to treat a simple file on your filesystem as a block device and do things with it that you would generally only do with a hard drive (which is seen as a block device.) So, you could set up a LUKS container inside a simple file, by setting it up as a loop file, creating a LUKS container on it, then creating a partition table inside it and multiple partitions inside that partition. All of this works because Linux is super strict with abstraction. For me, loop files, and their implementation in Linux, is sort of the pinnacle of great software design. The manual page for losetup, the command used to set up loop files, is informative and easy to read. Something like Tomb is possible only because of loop files. It is not supported on Mac OS because … well, of course anything useful would never exist on Mac OS. (I am forced to use a Macbook at work because of better Mobile Device Management support for that operating system, and MDM is a prerequisite for most IT teams: it allows them to control the company’s data and wipe the hard disk remotely if something happens to the laptop.) ↩
This happened in September 2022, and it was a soul crushing incident. It overtook my previous “I ruined my electronic device and I can blame only myself” incident from 3 years ago. ↩
If nothing has happened to you, that’s great. But it doesn’t hurt to be prepared, and that’s what I am evangelizing here. ↩
Now, I have no idea why this is required: Surely, changing the display has nothing to do with the software itself. But I have no idea how these devices work either, so I am not going to second guess the professionals. ↩
The process is convoluted, but definitely worth it. I copy HEIC files from iPhone to Windows through the Windows Photos app; then convert them from HEIC to JPEG using CopyTrans HEIC; and then, import them into Shotwell. Transferring photos from a digital camera into Shotwell is extremely simple: Connect the camera to your computer, open Shotwell, and click “Import All.” ↩
I did not use the cryptsetup CLI tool once when setting up encrypted partitions because the Disks utility on Debian (gnome-disks) works well with LUKS1. ↩

Review - The Book of Form and Emptiness (Ozeki)

2023-03-31T00:00:00+00:00

Rating: 4/5

Ozeki’s novel is good at the beginning and good at the end. The middle drags on for a little too long; I had at least a mild interest in figuring out how the characters end up. However, the interest levels get pretty low. It is a book about books, art, culture, the decline of reading, the increase of consumerism, capitalism, and the incredibly high number of things that are now a fixture in the lives of a few people. It is a portrayal of the present, as a dystopia, by a Book; that shows both the ability to buy things that jobs give people, and the precarious nature of those jobs in a society with no safety net. The novel is mostly sincere, but sometimes it crosses over into the cheesy. The novel has a character whose background is identical to Marie Kondo and whose philosophy is very similar (I guess) to Ozeki’s.

The most notable thing about this novel is probably the way that it is narrated. It has a character, the Book, which narrates the Boy’s story. And sometimes, the Boy chimes in and converses with the Book. The Marie Kondo character, Aikon, sometimes narrates her experience. The parts where the Book talks about itself, or about other books, are a bit cringe-y. It is a mixture of moral panic and cliches. It is hard to tell if this was an intentional choice. Here is a sample:

This state-of-the-art mechanical, computerized book-sorting system was installed as part of the renovation, and the books were beside themselves with loathing. They longed for human hands, for human touch. They bristled with ire at the indignity of their situation as they were spun, flipped, rotated, scanned, sorted, sent sliding down rackety gravity chutes into bins or hoisted hydraulically onto trolleys. It was more than any book could bear, and their lamentations rose above the clamor of the machines—We are not units! We, who once were sacred, next to God!

Ozeki’s book is really about Capitalism and Consumerism, the unlimited desire for things and the economic system that manufactures everything dirt cheap and then convinces everyone that they need this suddenly cheap stuff to stay at the top of a completely made up hierarchy. One of the homeless people that Benny hangs out with is German (a stand-in for Marx, perhaps?) and theorizes quite a bit (sometimes boringly). There is some purely theoretical, interesting rambling from another character:

It’s the fucking world we live in. It’s capitalism that’s crazy. It’s neoliberalism, and materialism, and our fucked-up consumer culture that’s crazy. It’s the fucking meritocracy that tells you that feeling sad is wrong and it’s your fault if you’re broken, but hey, capitalism can fix you! Just take these miracle pills and go shopping and buy yourself some new shit! It’s the doctors and shrinks and corporate medicine and Big Pharma, making billions of dollars telling us we’re crazy and then peddling us their so-called cures. That’s fucking crazy.

… but the most interesting part is the empathetic way in which Ozeki portrays the disgusting consumerism of her characters:

She [Benny’s mother] went back to the computer, typed in $50, and then sat back, feeling relieved. It was a bit of insurance. So as long as her maximum was the highest, she would still win. 3:59. Less than a minute left. She began to count down, and then, with only twenty seconds to go, the bid jumped to $32.45. And then it jumped again, and again! Not one, but two snipers were bidding on her Hansel and Gretel! She held her breath and crossed her fingers and counted–five, four, three, two . . . The Congratulations, You’ve Won! message popped on to her screen, along with her winning bid. $49.45. She sat back in her chair, triumphant. It felt so good to win.

Benny’s mother is a hoarder who is living in a house with a lot of things. She keeps buying things that she has no need for: books, craft supplies, snow globes. She keeps archives of old newspapers, from her job, inside the house. She doesn’t take out the trash. The picture of the house that is painted is astonishing, not very different from some of the worst kept houses that you would see on “tidying up” TV shows like Marie Kondo’s.

Speaking of Marie Kondo: Aikon is a Zen monk who is basically Marie Kondo, and also a stand-in for Ozeki’s voice of reason and (one presumes) beliefs. The similarity between Kondo and this character in the book is quite stunning. They both write a book, tell people how to fold clothes, arrange things, their books become a sensation in America, they star in an American TV show. Ozeki’s portrayal of Aikon does not feel lazy as a result though, because we never get inside Marie Kondo’s mind in her shows. Aikon comes at the “problem of too many things” from a completely different mindset: That mindset is refreshing. She lists the things that Ozeki talks about in this book as things that her producers asked her not to talk about:

Of course, the solution was quite simple: people just had to stop buying so much stuff, but when she mentioned this on a recent call with the American producers, their response was less than enthusiastic, and later they followed up with a memo, asking her not to talk about topics like that on the pilot. When Kimi [Aikon’s assistant] inquired what they meant by “topics like that,” they sent her a list: consumerism, capitalism, materialism, commodity fetishism, online shopping, and credit card debt. Speaking critically of such topics was un-American, they explained. American viewers wanted proactive solutions. Not buying was not proactive.

The final notable characteristic of the novel is the fact that things “speak.” They express themselves by hissing or shouting or creating a ruckus. They are silent when they are well cared for. And Benny is the only one who is able to listen to them. That is a pretty good idea. Benny is meticulously clean and organized. Undoubtedly, Benny is the hero, the ideal to aspire to.

I had not thought about things expressing themselves; I was assigning some feelings to them but not assigning a voice to them. (I have not looked at things as “just things” for quite some time now. Most of things I have I classify as tools, and been grateful about the ways in which they have made doing something easier.) The framework of thinking about things wanting care from their owners is powerful: It gets you to think about what a thing’s ideal state would be: why you should fold socks properly, why you should empty your bag each night. This deep respect for things and an acknowledgment of the role that they serve is important to me. By looking at tools as status symbols (“Do you have the latest iPhone?”) or just things to be “used and discarded” (“The headphones are around here, somewhere.”) diminishes them.

There is a very touching recollection of Aikon’s experiences working on relief efforts after the 2011-03-11 earthquake in Japan. I have lived in Japan for 4 years now, and I know that the 3.11 earthquake features prominently in the collective imagination of everyone who experienced that horrible disaster. The reminder that Japan is disaster-prone is everywhere: yearly drills, letters from the local government, news articles about disasters in other parts of Japan. But it is quite easy to believe that surely, you would not be affected by it. One must remain on guard against that complacency, as long as they live in Japan:

This was another important lesson in the impermanence of all things. Japan lies in a seismically active zone, so earthquakes are not uncommon. Disaster can strike at any moment, but we forget this, distracted by the bright, shiny comforts of our everyday lives. Wrapped in a false sense of security, we fall asleep, and in this dream, our life passes. The earthquake shook us awake, and the tsunami washed away our delusions. It caused us to question our values and our attachment to material possessions. When everything I think of as mine–my belongings, my family, my life–can be swept away in an instant, I have to ask myself, What is real? The wave reminded us that impermanence is real. This is waking up to our true nature. Already broken.

Other notable quotes:

Aikon talks about her book becoming a sensation:

Why was it that women could never work hard enough to quiet their nagging fear that they were not enough? That they were falling behind? That they could and should be better? No wonder they wanted simple rules to govern the way T-shirts should be folded, children raised, careers managed, lives lived. They needed to believe there was a right way and a wrong way–there had to be! Because if there was a right way, then perhaps they could find it, and if they found it and learned the rules, then all the pieces of their lives would fall into place and they would be happy.

Benny’s father talks about the moon landing:

It was 1969, and I was very young boy and worrying because in Japan, we have a fairy tale about a rabbit who lives on the moon, and I was afraid about the big American astronaut will hurt the moon rabbit. But everybody told me don’t worry! American astronaut is very kind man. He won’t hurt any moon rabbits! But still I was worrying.” “But it was okay, right?” “It was okay. We watched on the old black-and-white TV set because we have no Internets back then. Then we see the first astronaut, Mr. Neil Armstrong, go down the ladder to the moon and say some famous words–One small footstep for a man. One big jump for everybody else. Something like that. You know those words? Very famous.

Notes and Review - Lost Daughter (Ferrante)

2023-03-29T00:00:00+00:00

Rating: 5/5

One should never arrive in an unknown place at night, everything is undefined, every object is easily exaggerated.

This is a beautiful line. Every time I am booking a ticket to some new place, I remember this line. I remember the dread and uncertainty of arriving at night. Despite knowing that arriving at night is unwise, I arrived in Italy late one night in 2019. It was a timely reminder of my lack of wisdom; a line like this makes the lesson a memorable one. Seeing the confused characters in this book, my first instinct was to clamp down on their inability to decide and brand it rashly as immature indecisiveness; then, gradually, I would see the lens fog up and doubt creep in. Were they really indecisive, or do they appear unclear to us only in hindsight?

It was like a slight twinge that, as you keep thinking about it, becomes an unbearable pain. I was beginning to feel exasperated.

I watched the movie adaptation before reading the book. The movie was great. Dark and ominous right to the last scene, the unstated parts of the movie were exquisite; the setting was great and I felt the irritation caused by Nina’s family to Leda and Nina’s inexplicable presence among them. The first thing I noticed about the book was that all the characters are mothers, daughters, or soon-to-be mothers. Leda has 2 daughters Marca and Bianca. Leda talks often about her own mother. Nina has a daughter Elena. Elena thinks that her doll is her daughter, and she calls her various names. Nina’s sister-in-law is pregnant and expecting her first child. The doll is lost for a brief period of time; indeed, that is the central thread in the story. The doll is lost, and a long time later, it is found. During the intervening time, things get confusing. Leda is curious despite herself. She claims to have brought books and claims to want to read them, but she can’t take her eyes off Elena and Nina. Undoubtedly, Elena reminds her of herself: the young Leda; the unfulfilled scholar.

The one incident that haunts her is leaving her daughters for three years. She pursues the language of her scholarship, English. (There is an inversion in the setting of the movie and the book. In the book, Leda is an Italian woman and a scholar of the English language. In the movie, the situation is reversed: Leda is English and a scholar of the Italian language.) Recounting her experience of leaving her daughters, she thinks up this memorable paragraph:

After a while I heard a light knocking. Bianca came in, very serious, followed by her sister, timidly. Bianca took on orange from the tray of fruit, opened a drawer, handed me a knife. I didn’t understand, I was running after my own desires, I couldn’t wait to escape that house, forget it and forget everything. Make a snake for us, she asked then, for herself and Marta, too, and Marta smiled at me encouragingly. They sat in front of me waiting, they assumed the poses of cool and elegant little ladies, in their new dresses. All right, I said, took the orange, began to cut the peel. The children stared at me. I felt their gazes longing to tame me, but more brilliant was the brightness of the life outside them, new colors, new bodies, new intelligence, a language to possess finally as if it were my true language, and nothing, nothing that seemed to me reconcilable with that domestic space from which they stared at me in expectation. Ah, to make them invisible, to no longer hear the demands of their flesh as commands more pressing, more powerful than those which came from mine. I finished peeling the orange and I left. From that moment, for three years, I didn’t see or hear them at all.

The touching scene of separation is all the more jarring for we see her in the present, being treated coldly by her daughters, and we see the drive in her to continue her study. She doesn’t understand why she left them, and so she instinctively did not talk about it with her daughters. At the very beginning of the book, we encounter this sentence which explains why all she can do about this is write letters to her daughters:

At the origin was a gesture of mine that made no sense, and which, precisely because it was senseless, I immediately decided not to speak of to anyone. The hardest things to talk about are the ones we ourselves can’t understand.

If leaving did not make sense, why did she return? We get only the faintest of hints.

[Nina asks] “If you felt good why did you go back?” I [Leda] chose my words carefully. “Because I realized that I wasn’t capable of creating anything of my own that could truly equal them.”

This is the driving force of this novel: characters who do not reveal much; characters who would rather speak in cliches and through letters; characters who talk about their own confusions. They remain shrouded in a veil and we get rare glimpses into what they really care about. Why should we get any clarity about what these characters want, when we can’t get that kind of clarity with anyone else? Why should we get any clarity about what they want, when we can’t even be sure that we can get that clarity with ourselves? Ferrante is soothing and brilliant.

Other memorable quotes:

That’s what I was, superficial. And then that remark: children are always cause for worry. Said to a woman about to bring one into the world: how stupid. Always words of contempt, skeptical or ironic. Bianca had cried to me once between her tears: you always think you’re best. And Marta: why did you have us if all you do is complain about us? Fragments of words, mere syllables. The moment arrives when your children say to you with unhappy rage, why did you give me life: I walked absorbed in thought.

The Unstoppable March of Technology

2023-02-18T00:00:00+00:00

Every conversation that I have been in over the past 3 weeks outside the home has been about generative AI, and the vein of all these conversations is quite similar. Most of the people I met are optimistic about this new “technological advancement.”¹ I met a few skeptics who believe that generative AI is adept at generating only a summary of what’s already out there and incapable of coming up with something original.² I agree with those skeptics, based on the screenshots that abound on Twitter and my brief experience giving a handful of prompts to ChatGPT. The apparent inevitability of the “next thing” in technology is can not be understood as one that stems from rational thought: Rather, it is a cultural (almost religious) belief, and it pervades in the social psyche. For a while now, there has been this unstated acceptance that new technologies will come along, and those who don’t use them will be left behind in the dust. What if one were to ask the question, what if the dust is good enough? One wouldn’t even think of it; when has dust ever been “good enough”?. Is the cutting edge really something you want to be on? You better want to be there, or you will be left behind. Even if you are standing still on the travelator of technological advancements, you will be pulled along, dragging and screaming, on the unstoppable march of technology.

It is always hard to keep up with the news. I don’t read much of it anymore. Many times, I don’t know the topic that has been debated on all the newsletters for the past 3 days. I don’t feel like I miss many things. I find out about these things eventually, and most of the time, I am underwhelmed by what I find out. So, I have decided to not keep up with the news and instead spend my time on long form magazine articles, non-fiction books that try to guess where things went wrong³, and novels that have nothing to do with the present. But even this level of disconnection did not save me from finding out about the recent advances in generative AI technology: Large language models.

For a long time, the goal of AI scientists was to make something that was sentient and could think and do things like a human being. Some AI scientists thought the creation of such an AI system was inevitable, so they started working on the problem of ensuring that these AI systems (which will undoubtedly be created, duh.) don’t end up adversely affecting humans. Such an AI system does not exist yet. However, AI technology has come a long way: It can now write a mediocre essay about any topic, which is probably plagiarizing 100s of great essays and millions of mediocre essays on the same topic.

Another feature of this generative AI craze has been that it can write individual, isolated functions in programming languages for things like integrating a spreadsheet with some other tool or for connecting to the API of a service and automating some task, which was being performed manually. This feature is probably not very scary to most software engineers; nevertheless, let me say it aloud just once: Most engineers are not writing isolated functions. Their everyday task involves making sense of the myriad ways things are done at their workplace, and figuring out a way to make the technological stack a little bit easier to understand, while convincing a large retinue of coworkers that what they are doing actually makes things easier. Further, this kind of “duct-taping” between spreadsheets and some other tool is only necessary because most people agree that the tools that they use daily at work are terrible in general and particularly bad at achieving their primary function.

Generative AI can also create some images. Wired magazine thinks that these images are better than “art created by most humans.” They declared that generative AI now has “lowercase creativity.” I think this claim is wrong. AI art is AI art; not “human” art. There is no “authority” who can decide what is “real” art and what isn’t. So, whatever each viewer decides goes. For me, a painting by Klee or that of a minion by my niece is still objectively and incomparably better than the images generated by a computer program. Despite all the dressing up of AI as some “magical system,” it is just a computer program.

Funnily enough, even this article about “AI’s ability to generate images that are better than art by most humans” is not content with this bizarre and monumental claim. Somewhere in the middle, and completely out of nowhere, the author starts talking about how in the future you could generate 3D images, then use that to create a world in the metaverse and then walk through it with your VR headset. (You might notice that all the terms used in the previous sentence were coined in the 90s and entered the public’s psyche less than a year ago, when one company decided to change their name, and make one of those terms the central theme of their future businesses.) A simpler reading of the article is that it was written by somebody who has one foot in the door of every single hype train that is leaving the station.

Once, there were letters and we used to wait. Then, there was the telegraph. Then, radio, television (Idiot Box), computer, the Internet (Revolutionary), smartphones, social media (Anxiety), connected people, and connected homes (Internet of Things). Every technology brought along some problems which were fixed by the next advancement. No one stopped to think whether the adoption of each technological advancement was necessary; whether it introduced more problems than it solved; whether this march of technology was really inevitable and whether it was really a march forward.

Freud noticed this early:⁴

Freud once pointed out that new technologies merely solve problems created by other technologies. To the common refrain that without the telephone, we’d be unable to hear the voices of our adult children who live hundreds of miles away, he replied, “If there had been no railway to conquer distances, my child would never have left his native town and I should need no telephone to hear his voice.”

Neil Postman, a social critic from the 1980s, spoke about it too:⁵

To be unaware that a technology comes equipped with a program for social change, to maintain that technology is neutral, to make the assumption that technology is always a friend to culture is stupidity plain and simple. Moreover, we have seen enough by now to know that technological changes in our modes of communication are even more ideology-laden than changes in our modes of transportation. Introduce the alphabet to a culture and you change its cognitive habits, its social relations, its notions of community, history and religion. Introduce the printing press with movable type, and you do the same. Introduce speed-of-light transmission of images and you make a cultural revolution. Without a vote. Without polemics. Without guerrilla resistance. Here is ideology, pure if not serene. Here is ideology without words, and all the more powerful for their absence. All that is required to make it stick is a population that devoutly believes in the inevitability of progress.

– p.158, Amusing Ourselves to Death (Postman)

Postscript: A list of articles that take a sober view of what is lost by always being on the cutting edge:

It is interesting that the change has been characterized as an advancement even before it is completely clear what the change really even is. ↩
This undoubtedly depends on what you mean by “original,” on which there is considerable debate. ↩
We are all in tacit agreement that things have gone terribly wrong, right? ↩
Author: Meghan O’Gieblyn. From the “Dear Cloud Support” column in the February 2023 issue of Wired magazine. ↩
I found this quote on 2023-04-29 when I was re-reading Postman’s book “Amusing Ourselves to Death.” Postman used the same word, “inevitable,” as I had chosen to use in the opening paragraph of this post. I am glad to find out that others share the frustration that I feel when people talk about “unavoidable technological progress.” ↩

What To Do With All This Media Theory?

2022-12-10T00:00:00+00:00

There has been a lot of media theory talk over the past year in the mainstream (newspapers, news websites, television) and on the intellectual sidelines (podcasts and blogs). The discourse is heavy on the origins and consequences of the continuing, effectively unstoppable, decay of communication over traditional media. (Savvy authors might use terms like “signal to noise ratio.”) However, all this discourse is suspiciously devoid of any advice for us, the spectators. Repeatedly, there is the defeatist assertion that most people will probably be addicted to their screens despite knowing how the screens manipulate them. I don’t think this defeatism is necessary. What should people do to avoid the consequences that authors are expounding on? Should they try to cut their information intake? Should they go “off the grid,” a concept that Opinion writers have made a cottage industry out of talking about?¹ The discourse has no response. Here’s my non-defeatist take.

A Rant

The premise of this whole post is that current media theory doesn’t offer solutions or even lighthearted pointers towards a better way to use our tools and consume media. In fact, the most one can probably get from it is that we are in a transition period and we shouldn’t worry because everything will get sorted out, by someone, somehow, at some vague point in the future. That advice is not good enough. My belief is that theory and discourse should not merely theorize about reality; it should put forth recommendations for readers about the methods for applying realizations in real-life. And the action has to happen now.

Before presenting the defeatist examples, let me say outright that the people giving the defeatist takes are the most informed on the subjects and I respect their work, because I have learn most of what I know from their work. Disclaimer aside, a couple examples of the defeatist takes from people who talk about media:

a lot of us, most of us, maybe are going to and do spend a lot of time staring at a screen, skimming with distractions everywhere

– Opinion - Transcript: Ezra Klein Interviews Maryanne Wolf - The New York Times

As I began reading the novel I’m now close to finishing, it was difficult for me to get lost in the story. When I do read something that is intellectually demanding, I find my mind wandering from the page after only a few moments. I assume that by now this is just how most people feel about reading and that some may not even know that there is any other way.

– Reading As Counter-Practice - by L. M. Sacasas

If the internet is as transformative a technology as the printing press, and I think it’s certainly comparable, then it’s going to take several decades to fully adjust to the changes it has wrought.

– Transcript: Ezra Klein Interviews Sean Illing - The New York Times

I started writing this post after listening to Maryanne Wolf on The Ezra Klein Show. Wolf is eloquent throughout the show, and she spoke with the zeal and enthusiasm of a real ideologue. She really believes in her work, and her work was magnificently revealing about the ways our reading habits on screens and on paper are intertwined with each other. As I mentioned above, the ways in which parents’ addiction to screens affects children is another part of the conversation where her expertise shines through and she articulates things with amazing clarity.

Towards the end of the show, the host asks her a simple question: “What should someone who wants to become a deep reader do to regain the mythical state of deep reading that Wolf advocates for?” Here, Wolf falters. First, she shares her habit of spending 20 minutes after waking up and before going to sleep reading a book, philosophy in the morning and classics at night. The host pushes back on her suggestion. Then, she suggests that it is up to each individual. Then, she digresses from the topic of reading to the phenomenally more complex topic of “just being yourself” using other mediums such as music. This was a disappointing end to a great conversation.

L. M. Sacasas’ newsletter is rich in media theory but light on recommendations, and perhaps it is so by design. Illing’s conversation about their new book about the connection between media and democracy was also informative: It touches back to the origins of media theory during the post-war period, when the TV was initially introduced. Illing is optimistic about the tools we have today but thinks that we are only at the beginning and that we don’t need to really worry too much about the bad ways in which these tools are affecting us, because a lot of change is yet to come.

The lack of recommendations might also be from the structure of these conversations and the medium through which they reach their audience: The News. The self-inflicted wound of saying consume less news on a newspaper is not beyond any of these experts; however, I doubt that they would say it. The closest I can think of is something that Ezra Klein said in his podcast with Johann Hari about attention, in which he says that we should consume far less news than we do.

The Constants

Here are the constants. These are the things that are referred to as “required reading” in most of the conversations, and are rarely debated as being true.²

Medium is the Message
1. McLuhan wrote a book about this: “Everything on T.V. should be entertaining. Education shows like Sesame Street are on T.V. Kids who watch TV end up believing that education should be entertaining too.” It’s not that they don’t learn something on these “edutainment” shows; they do learn something useful and this complicates things furthers, because they now expect that education should be presented to them in an entertaining manner. (Remember all the “Let’s make Mathematics fun” talk? Duolingo tries to make language learning fun by gamifying it.)
Attention should be conceptualized as a resource. Attention spans are decreasing.
1. Attention span is (roughly) the period of time for which someone can pay undivided attention to a single concept or thought. Attention as a resource is a way of conceptualizing what is being sold by Facebook to the advertiser, when they provide the Newsfeed for free. There is no monetary exchange. The only exchange is temporal and so time itself (or attention, the act of spending time on something) is the resource.
Reading is in decline

The percentage of 12th graders who read a book or a magazine every day declined from 60% in the late 1970s to 16% by 2016, and 8th graders spent almost an hour less time watching TV in 2016 compared with the early 1990s

– Trends in U.S. Adolescents’ media use, 1976-2016: The rise of digital media, the decline of TV, and the (near) demise of print. - PsycNET
Mass media is designed around consumer buying. Sensational coverage gets people more engaged
1. This has been a known trope about television coverage of terrorism, protests and accidents, or other traumatic events for a while now. Every protest produces the same set of photographs: A group of people, preferably in masks, throwing dangerous objects like rocks and Molotov cocktails at armed police officers, preferably in riot gear. All of it is because of the economic absurdity of covering something without sensationalism. This video illustrates this point.
Content is designed to be engaging and sensational, not informative or educational
1. I wrote a series of posts analyzing the home pages of several news websites at the same point in time. The Nancy Grace section in the TV show Newsroom is indicative of this tendency.
Information is super-abundant on the Internet. Algorithms sort through this information and act as the middlemen who deal content to users.
1. Long gone are the concepts of a trusted TV stations or a single newspaper of record. There are innumerable sources of information on the internet and anything you want to believe can be proved by picking the sources that support your viewpoint and smearing those that oppose your beliefs
Algorithms are designed to increase engagement and profits through advertising networks
1. Many news websites have advertising through ad networks, or pleas for donations on all of their articles. Clearly, “journalism” is not paying for itself.
Some people hack³ the media by being egregiously engaging and sensational
1. Trump, Musk, Kangana Ranaut
Twitter is not the platform for informed discourse
1. Saunders, who wrote the Braindead Megaphone, says that Twitter is a strange place because it places a limit on what can be said and how many characters can be used. Why is this limit being placed? See “(1): Medium is the Message.” What can be said in 280 characters is probably neither useful nor insightful; it is merely provocative. (Typical example of click bait that is now accepted: “Why did Elon Musk buy Twitter? A thread. 1/n”)
Twitter is the platform for informed discourse
1. Ezra Klein has said on his podcast repeatedly that Twitter is an important place for journalists because that is where they get their story ideas and that is where they realize what the “vibe” of a particular subject really is. Clearly, most journalists think that Twitter is a good platform, despite vehemently denying its validity and arguing about the ease of being manipulated into believing something is more important than it actually is.
Movement and vivid colors capture people’s attention
1. This is a dig at some of the more obvious takes around parenting. Children don’t come into possession of video games / gaming consoles / digital screens in a vacuum. Parents buy these things for their children to achieve some goal. This goal might very well be to fan the parents’ addiction to media and screens, an addiction that is not good for either parent or child. Wolf’s expertise with children, literacy, and the effects digital skimming culture have on parents and children is another very good part of the conversation.

The Actions

Here are the actions that I took over the past few months.⁴

Less Skimming. Less News.

Stop reading the news. You don’t need to know everything.

Okay. I am going to rip the band-aid off real quick: 90% of News is either sensationalist or useless.⁵

I used to be a big-time news reader. I read a morning briefing from the New York Times. I listened to about 40 minutes of news podcast on an average day. I spent a big portion of free time during the day reading articles that appeared in the notifications on my phone. The constant flow of news powers the adrenaline rush of being part of a (hyper) connected world. However, it has no practical use. I don’t work in the news or any news-adjacent business, and I have no obligation to be informed of “current affairs.”

Now, I listen to 20 minutes of news, from a briefing podcast, every morning. This has been my only source for the past 6 months and I am doing just fine. I will not be returning to reading more news anytime soon.

A caveat: I am yet to completely think through the impact this has on my beliefs about democracy itself: I have said before on this blog that an informed electorate is a prerequisite for a democracy that works. I look forward to reading Illing et al’s work about democracy and media consumption and trying to have clearer thoughts on the subject.

More Magazines

Buy the print version of a non-news, non-“current affairs” magazine

I used to be completely disconnected from magazines. I did not appreciate the form until last year, when I subscribed to the New Yorker on a whim. While that particular magazine published at a rate that was much too fast for me (4 issues / month), I subscribed to the print version of WIRED, a tech- and media-adjacent magazine this year (1 issue / month). I really like the format: The shorter columns are great for free time during the day, the longer reads are thought-provoking profiles or excerpts from books. There is almost no politics in the whole magazine. Topics are not limited to current affairs. (My favorite long pieces from the last few issues was an eccentric profile of the filmmaker Taika Waititi and a fascinating interview with one of the authors of the book The Dawn of Everything⁶.)

A note: Magazine aggregators like the Apple News service might look inexpensive. I suspect that having access to that much information can tempt readers to switch back to the digital skimming modes of reading. So, for now, I am staying away from aggregators and algorithms.

More Fiction

Good fiction is a great story with interesting people in it.

I think fiction is where the most amazing type of reading happens. Regrettably, a lot of non-fiction is little more than information gathering or propaganda for some hustle culture concept like Todo lists or habit forming or how CEOs spend their day. Only the absolute best non-fiction books⁷ rise above the “merely communicating information” level to the “mindset forming” level.

A note: A lot of non-fiction, especially the “top of the best seller lists” non-fiction books, are little more than a single, interesting insight repeated over the entirety of a 200-page light read: These are the airport best sellers that you should stay away from. This podcast pokes fun at these books and is entertaining.

More Blocks of Time

Reading for 2 hours at a time is better than for several periods of a few minutes each

This was my most useful insight of the past year. Deep reading (or the focus required for that) doesn’t just happen. It is also very hard to maintain that focus consciously. So, the best method to naturally prepare yourself for a period of deep focus is to set aside a large block of time and do nothing except read in that block. This is hard, and perhaps many weeks will go by until I get a satisfying block to do this. Nevertheless, I have resorted to this approach now.

For something as open-ended as reading, the levels of motivation vary over time and the books that you pick up; chances are high that you can turn your “desire” to read into a “habit” of reading through any sort of strict framework. Coming up with the framework and enforcing it on yourself though? Well, that is on you, the reader. The hardest part of doing more “deep reading” is that there is no trick. But that should not prevent media theorists from putting forth better recommendations.

“I went into the mountains for a week. I was completely off the grid. There, I had the amazingly original insight that we are addicted to our phones.” ↩
For some of the constants I have added some notes in the list about where the concept comes from and how I understood it. These notes are completely optional and you can skip them if the constant is self-evident or obvious to you. ↩
i.e. manipulate. The media people are vehement that they are being manipulated. ↩
My reading about media and media theory is only beginning. I hope to revisit Postman’s book about television, the recent book by Illing and co. about the connection between government and media, and a few research papers that Wolf mentions in her conversation. ↩
There is a syntactical fight about what “useful” really is and who gets to decide it. In this case, I get to decide it because I found that most of what I was reading in the news was not useful except as trivia. ↩
The interview might be fascinating for this single line: “Wengrow had no contact with Musk, about whom he appeared to know little, and care less.” ↩
In my opinion: Mokyr’s book about technological progress; Rosling’s book about the world getting better; Piketty’s book explaining how economic inequality should be measured ↩

Notes and Review - The Castle (Kafka)

2022-12-02T00:00:00+00:00

This novel is humorous. The characters and their interactions, the presumptions that they make about each other based on trivial details, the protagonist’s (His name is K.) strange obsession over every single alphabet in the letters that he receives, characters that have a job which they do only when they want to. Everything in the novel is humorous. On the surface. Just underneath the surface, lurks the boundless struggle and omnipresent hopelessness of a life in the world that K. inhabits. (The world that he is in is eerily, and perhaps intentionally similar to our own.) This is not a dystopia; the bureaucratic hell that K. is subjected to is one that many are familiar with. The nightmare does not stop when you have procured every new document that the government has ever issued to its citizens; it continues, for the government takes great relish in moving the goalposts and confusing people; keeping them inside their Web browser until their citizens are exasperated enough to just close the tab and move on. The only difference between the citizens of our world and K. is that K. does not move on.

Kafkaesque Imagery

As always, Kafka’s imagery remains unchanged. The first room that the protagonist enters is dark and suffocating:

It was a large, dimly lit room. Coming in from outside, he could see nothing at first. K. staggered and nearly fell over a washing-trough; a woman’s hand caught him. He heard a number of children shouting in one corner. Steam billowed out of another, turning the twilight into darkness. K. might have been surrounded by clouds

Officials in Bed During Hearings

This is one of Kafka’s classic images. I believe that his intention with showing officials in bed is to show their lethargy during work and their lack of motivation to really help the petitioner. During the last section of the novel, K. meets an official who hints that he might be partly responsible for K.’s case and that he can grant any of K.’s wishes on the spot. He says that the officials are overwhelmed with work and so sensitive that they can barely sleep at night and thus, they conduct all their hearings in the Castle Inn. But the Castle’s staff thinks that the Officials sleep too much, and in fact, it is because they have so much work during the day/night that they are always tired.

All of this contradiction and absurdity, the midnight hearings, officials in bed, people being asked to wait in the cold to meet officials, official summons that arrive through messengers, assistants who are not assisting, serves to make K.’s situation look more and more hopeless and pathetic.

As the reader, I wanted K. to just give up, on many occasions. But therein lies the rub: When I was stuck in a bureaucratic hell, I did not give up. I would not have even imagined it. To fight your way out of the mess that you are in is the only available option.

Officials Don’t Make Mistakes

An important moment in the novel is when K. finds out that he is in the village by mistake. Kafka’s writing is deliciously absurd. I was giggling throughout this long section in which an official explains how officials never make mistakes, and thus when a document appears in front of them, it doesn’t occur to them that it was due to a mistake, and thus, due to their assumption that they don’t make mistakes they end up making mistakes and what should have been ended in a minute of honest doubt and humiliation drags on for years.

[The official who is speaking explains that the proposal for appointing a land surveyor was sent to his department by department A. Then, the monologue continues.]

For a start, however, I can tell you the story without files. We replied to the decree I was talking about by sending our thanks, but pointing out that we didn’t need a land surveyor. However, that reply doesn’t seem to have reached the original department, let’s call it A, but by mistake went to another department, B

as often and understandably happens, and indeed should happen, in view of the meticulous nature of all the official work done, the head of department was relying on us to send an answer, whereupon he would either appoint the land surveyor or, if necessary, correspond with us on the subject further. As a result, he neglected to look at the preliminary notes and let the whole affair lapse into oblivion.

It is a working principle of the authorities that they do not even consider the possibility of mistakes being made. The excellent organization of the whole thing justifies that principle, which is necessary if tasks are to be performed with the utmost celerity

When asked a simple follow up question by the protagonist about whether the supervisory authorities who are supposed to catch such mistakes did not succeed in their duty this time, the official comes up with this gold:

Only a complete stranger would ask your question. Are there supervisory authorities? There are only supervising authorities. To be sure, they’re not intended to detect mistakes in the vulgar sense of the word, since there are no mistakes, and even if there is a mistake, as in your own case, who’s to say that it’s really a mistake in the long run

The official also describes something that has happened to me once when dealing with the government:

And now I come to a particular feature of our official mechanism. When an affair has been under consideration for a very long time, and even before assessment of it is complete, it can happen that something occurs to settle it, like a sudden flash of lightning at some unforeseeable point, and you can’t pinpoint it later. The case is thus brought to an arbitrary, if usually quite correct, conclusion. It’s as if the official mechanism could no longer stand up to the tension and the years of attrition caused by the same factor, which in itself may be slight, and has made the decision of its own accord with no need for the officials to take a hand.

K.’s summary of all this? Pretty good:

K.: So the outcome is that everything is very confused and nothing can be solved, and I’m being thrown out.

Mayor: Who would venture to throw you out, my dear sir? The very lack of clarity in the earlier questions assures you of the most civil treatment, but you appear to be oversensitive. No one is keeping you here, but that doesn’t amount to being thrown out

So, you are not being thrown out. But everything is very confused and nothing can ever be solved.

Hierarchy is Everything to the Officials

Klamm is a God-like figure to the people in this village. K. is engaged in a tussle to meet with this man. When he tries to do so, he is thwarted. Soon after, Klamm escapes quickly and K. and an official observe him through a keyhole. (Yes, 2 grown men peeping through a keyhole at Klamm.) After this plays out, the official gives a summary that is indicative of the absolute reverence that one must show when they are in a hierarchy:

K. asked: ‘Has Klamm left, then?’ He said it not to have what he already knew confirmed but to anticipate an attack, for he rather feared he was vulnerable now. The landlady walked past him in silence, but the gentleman said, from his little table: ‘Yes, to be sure. Once you had left the place where you were standing guard, Klamm was able to go out. But it’s amazing what a sensitive gentleman he is. Did you notice, ma’am,’ he asked the land- lady, ‘how nervously Klamm looked around?’ The landlady did not seem to have noticed, but the gentleman went on: ‘Well, luckily there was nothing left to be seen. The driver had covered up the tracks in the snow.

This is the first point at which K. shows some defiance. This gentleman is a secretary of some sort to Klamm. He is working on his report for Klamm, and he wants K. to answer some questions so that he might be able to complete the report. But K. refuses to be questioned. There is no way to know whether this is a turning point in K.’s story. Kafka’s implication is probably that answering his questions/refusing to be questioned will have little impact on the final resolution of K.’s case. (And when will that come? There is no final resolution to expect.)

Defiance and Dejection

On the topic of defiance, the second protagonist of the story is Amalia, one of the messenger’s sisters. The messenger and their family is reviled in the village. People scoff at them, and they treat them quite poorly. K. and the reader find out, when Amalia explains, that this is because Sortini (an official) called Amalia to him and Amalia refused to go. Klamm and Sortini (and other men associated with the Castle) are philanderers who call women to them at periodic intervals. The landlady and even K.’s partner, Frieda, were women who had been called by Klamm. Amalia’s defiance to submit to Sortini’s (by extension, The Castle’s) wishes made her and her family a pariah in the village. It crushes her parents, and they instantly become too old and frail and need Amalia’s constant care.

The curious thing is that the Castle never gets officially involved with punishing Amalia. Indeed, no punishment is warranted because she did not do anything that was against the law anyway. She simply refused to answer a summons from a private individual, who happened to be associated with The Castle. However, there is a general sense that Amalia has done something wrong. The story keeps changing and becomes more and more extreme with every retelling (Chinese whispers).

When the stories have done the round, the Village is ready to forget and simply expects them to return to their earlier way of life. However, they don’t know how to do so. So, the Villagers continue to believe that there is some sort of punishment and avoid them forever. When the family petitions the Castle to help them, the officials in the Castle (understandably) ask them what they must help with? What have they done? What should they be forgiven for?

It is futile to ask for forgiveness for something when you have no idea what happened or what your mistake was. The story of the Barnabas family is one of defiance and dejection. They had done nothing wrong. But The Castle controlled everything. Even the appearance of defiance against such an absolute authority has a high cost. Despite their story, I don’t think that Kafka urges the reader to comply. I think Kafka urges us all to be defiant of the bureaucratic hurdles that we face. But I think he asks us to remember that “The Village” will forget at some point and we must also forget then. We should not let our defiance weigh too heavily on us; so heavily that it prevents us from returning to normalcy when everyone else expects things to “go back to the way they were.”

This special case of apparent external normalcy coexisting with internal turmoil is a favorite of Kafka’s, and he returns to it in both his other novels (The Trial and Amerika.)

Little Tyrannies

Kafka is undoubtedly obsessed with authority. Especially the kind of authority that is conferred upon people who are “big fishes in small ponds.” Throughout the novel, the landlady has her own little tyranny inside her inn, Frieda has a small one when she is barmaid, the secretary has one when Klamm is not around, Klamm has one that is all-encompassing and covers the whole village.

The existence of these tyrannies and the attempts of the tyrants to exhibit their power inside their little tyranny is the root cause of the troubles that K. is facing. Indeed, it is the root cause of the troubles that many of us face when we deal with intractable bureaucracies. How can one deal with these tyrannies? Kafka gives us a clue:

That’s the way it’s organized; all the gentlemen from the castle have their village secretaries.’ Momus, who in fact had been listening much more attentively than K., added, to the landlady: ‘Most of the village secretaries work only for one master, but I work for two, Klamm and Vallabene.’ ‘Yes,’ said the landlady now, also remembering, and turned to K. ‘Mr Momus works for two masters, for Klamm and for Vallabene, so he is village secretary twice over.’ ‘Twice over–fancy that,’ said K., nodding as you might nod to a child whom you have just heard praised and addressing Momus, who now, leaning forward, looked up at him. If there was a certain disdain in that nod, it either went unnoticed or actually seemed requisite. The merits of a man from Klamm’s close circle were being presented at length to K. of all people, deemed unworthy even to have Klamm set eyes on him by chance, and it was done with the unconcealed intention of demanding K.’s recognition and praise.

Kafka indicates that we should ignore the empty boasting of tyrants and be defiant in our demands. Things often resolve themselves, as the mayor explained earlier, and the resolution will have absolutely no connection to your efforts to solve the problem or the tyrant’s efforts to block a resolution anyway.

Chaos, Confusion, Mania

There are 2 particular scenes which are remarkable for the amount of chaos and confusion that is conveyed in them:

K. and crew wake up in the morning in the school building: After having been appointed the school janitor, K. proceeds to sleep in the school building. When he wakes up in the morning, he sees himself surrounded by children waiting to enter the school building and jeering at them. Then, confusion ensues as they rush around to clean the school building and try to get the place in order. There is a huge cat that is bothering them. His assistants are creating their own chaos in parallel. Reading this scene really feels like the author is trying very hard to put us in a trance and convince us of the hopelessness of K.’s situation.
K. meets with Erlanger and the files are subsequently distributed to all the officials who are in the basement rooms at the Castle Inn. Once again, the situation is one of utter chaos and mania. Files are being passed around and kept at the doors of officials. When the files are kept, officials greedily grab them. They ring the bells to call someone. This exercise ends after a long description of the craziness that has ensued during a procedure that one would assume was mundane, especially since it was being done every morning. Nothing is mundane in Kafka’s world.

Why Do You Read Entertainment Supplements?

2022-10-13T00:00:00+00:00

The newspaper represents the canonical form of media. This medium is devoid of the strangeness inherent in 24-hour TV¹, and has been around long enough for its form to reach a state of equilibrium; a state that can be understood. Of all the pages in a newspaper, I want to talk about the entertainment pages. What is the point of the entertainment supplement that accompanies newspapers? These supplements often start with a headline in which an actor is promoting a movie. The key however is that the actor says something unrelated; or “reveals” a secret about their work or life so that the reader is first trapped into reading the article. Eventually, the reader will find out that they are promoting a movie. I did not know this for a while; when watching television shows where guests would come on the show as “judges,” I was not able to recognize immediately that the people who came on the show were there only when they were promoting something. Once I realized that actors go on TV shows for only that reason, the connections were much easier to draw. Whenever a new movie comes out, the people associated with the movie try to get as much footage as they possibly can. This is simply the way marketing works; a banal truth. Is this media blitz anything except marketing? Why do the media organizations become tools in this marketing? Why do people fall for this not immediately obvious marketing ploy, despite years of formulaic use? Those are the questions I am going to attempt to answer in this post.

Page 3 is A Waste of Time

Let’s start with the apparently easy question. Newspapers contain “Page 3”, the page that has a bunch of photographs of the parties that happened last night. Page 3 is useless for everyone who is not someone who wants to go to these parties or is trying to make a career as an actor. The information contained in these pages is mostly about who is hosting parties, why, and whom they invited. By looking at these lists, perhaps an aspiring actor can identify the people that they need to get in with to be cast in the next big-budget movie that comes along.

Entertainment Pages Contain Only Trivia

What about pages 1 and 2 of the entertainment supplement? Is there any information in these pages at all? In the most basic sense of the word, there absolutely is: Actors talk about the pranks that they played on other actors when they were on set; a newbie talks about their experience working with an experienced actor; an experienced actor talks about working with a reputed director; and so on. But this information is seldom being communicated for its own sake. It is being communicated as a ploy to sell the image of the speaker to the next person who is going to employ them. For example, the actor that talks about a director allowing them creative freedom with dialogues is expressing that the actor expects this freedom to act well. This piece of information matters only to someone who will be in a position to choose the actor for some role. For everyone else, it is just trivia.

(A non-trivia usage of this information also exists. This information, in some cases, might allow the viewer to assign value to the actor’s or the director’s skill and decide whether they trust them to deliver a good product in the future. However, I don’t believe that the information communicated in entertainment supplements can sway people much. By definition, this is information that is designed to paint the actor in a good light for a future producer or director. The information is never delivered along with the measured criticism that one would find in any good review; it comes only with a modest amount of self-criticism.)

By this logic, for someone who is not in the “film industry,” the entertainment supplement is simply a collection of trivia about people whose films (or other artworks) were released recently.

Is There Such a Thing as Useful Trivia?

There is a lot of trivia out there. The media industry is built around telling you about it. I think that trivia is mildly interesting in the party setting. It can keep the conversation going when you can talk about how some actor is famously down-to-earth. (Why shouldn’t they be down-to-earth?)

For me, trivia isn’t really “useful” information for its own sake. “General knowledge”, that set of facts which includes such diverse things as the name of the second person to walk on the Moon and the national flower of Vietnam, isn’t useful for its own sake either.² Trivia is useless unless it is being used to do something, such as deciding whether or not you want to cast an actor who always has a prank up their sleeve in a historical drama. (You probably don’t, but what do I care? I will never be in that position anyway.)

When the actor/director talks about their craft and their practice of it, things get interesting and veer out of the “trivia” zone.

For example, here is a fact about Anthony Hopkins³: “Hopkins reads the script 200 times or so, ensuring that the dialogue is lodged in his brain.” If you have watched the TV series West World and you noticed something different about his acting compared to the other actors, perhaps this could be one reason why he is able to deliver in a way that other actors can’t.

Another example is trivia about a director’s choice⁴: Eric Heisserer, screenwriter who adapted the short story “Story of Your Life” by Ted Chiang to the script for the movie Arrival (2016), told Chiang that he wanted to change the context of his story before adapting it, because he felt that Chiang’s framing of the main character’s dilemma (in the short story) was not convincing enough to engage a visual audience. This has huge ramifications on the final piece of art. I believe that this single change completely changed the movie adaptation from the short story: The message communicated in the movie is the exact opposite of what Chiang communicated in the short story.

Let’s assume that artists would love to keep talking about their craft. This might still not be interesting enough for the media industry to sell newspapers. How many newspapers can be sold on the above non-trivia facts? Compare your estimate to the number of newspapers you think can be sold when one actor claims that another actor plays a prank on all his colleagues which involves a bucket of water (or some other “prank-y” object.) My estimate is that the latter fact can sell manifold more papers. The entertainment supplement headlines write themselves when they are about the “atmosphere on set.”

This is the reason useless trivia often finds itself on the front page of entertainment supplements (and gossip sites or even serious news sites on the Internet.)

Buffoonery Disguised as “News”

Once people realize that they are being suckered into buying things by these salesmen disguised as artists and that they are being fed an endless series of unimportant and useless drivel, wouldn’t they simply stop buying newspapers? No. All of this buffoonery is disguised as “news,” and to not follow the news is sacrilege. After all, you want to be an informed citizen, don’t you?

The ridicule that someone is subjected to for not knowing the president of a country is not for the fainthearted. The news has replaced every other way to pass our time as the only thing that is worth our attention. Push notifications about the latest protests in Iran or the newly elected leader of a faraway country keep everyone hooked to their screens. I have felt this when I see real time stories unfold. It is hard to get yourself out of the situation where you feel like you have to know what is going to go down; even though you might not lose much by simply following up on a news digest in a few hours / days / weeks.

On the morning of February 24, 2022, I found myself in front of live news coverage from Al Jazeera on YouTube. The Russian invasion was about to begin in a few hours; however, the correspondent was walking around a Ukrainian town and talking about how people were surprisingly normal; how the atmosphere really felt like the “calm before the storm.”⁵ A knee jerk reaction is that this is preposterous; the citizens must not have been informed; they must have been watching state propaganda; some form of “I would not be this calm if I was in Ukraine”; etc, etc.

If I avoid that knee jerk reaction, then I see that this is the most likely situation in most places. If a war is about to start but no one knows that it certainly will, why would they change their behavior? It is the observer who is projecting their expectation onto these people. It is these people that we see on the screen who are actually living the experience the observer is watching from far away.

Here’s another question: Can this coverage be educational? Does this teach other people how to behave when their city might be invaded? No, I don’t think so. 7 months into the war, coverage of that first day of the war seems misguided and naive. Nothing useful could have been learned on that first day. Everything that was important was either far back in history (e.g. Russia started sanctions-proofing its economy in 2014; Germany was highly dependent on Russian natural gas) or about to come (e.g. Russia would turn off energy supplies to Germany; Ukraine would not share intelligence with its Western backers). All of these things are contained in history books and other kinds of outlets which focus on creating evergreen content, such as Vox or Vice. They provide a larger perspective; the perspective that one would find educational.

Haven’t Things Always Been Like This?

No, they haven’t. A century ago, things were not like this.

(I make this assertion with full knowledge of the human folly of thinking that the situation that we find ourselves in is completely novel and has never been dealt with before. I am probably making the same mistake here. People who were alive when printed books were getting exponentially cheaper or when literacy was increasing rapidly probably also felt that things were getting much worse than ever before in history. Indeed, one of the first effects of the printing press was in the spread of Protestanism which sparked many wars.)

Downton Abbey, a show based in the early 20th century, shows a doctor who is not confident about the latest treatment for one of his patients’ illness. This is unthinkable in today’s world of medicine, where doctors are expected to know everything in their field of expertise. Until the advent of cheap telegrams and local newspapers, which started printing national and international news, it was still possible to know only the news of the locality that one lived in.

These changes can happen fast because they are simple first-order effects of technological progress. And as Illing notes, technological advancements like the telegram, television, and the Internet changed our society fundamentally without a vote or the majority’s consent. Our belief that technological progress is inevitable is enough to cement these advancements into our way of life. An individual can merely adapt to this new way of life. (Case-in-point: All the recent talk about the metaverse obscures one question: Who wants the metaverse?)

Summary

People buy newspapers because they feel it is their job to be informed
Society hammers into children the importance of “general knowledge.”
Newspapers need content which is interesting at a superficial level and entertaining in a boring, obvious manner
People in the entertainment industry provide this content in the form of trivia which they hope will impress their next employer and get them their next assignment
Media organizations peddle this trivia around society
Society makes those who don’t know the trivia look uninformed
Consequently: People read entertainment supplements, despite getting the worst part of the bargain, and losing time in the process

I read a relevant quote before publishing this post in a related newsletter.

It is not just the fourth page of newspapers that is made up of advertisements. The whole body of the paper is a one big continuous and general advertisement.

– Original: Gabriel Tarde (a French sociologist); Quoted by Citton in Ecology of Attention; Quoted by Sacasas in The Pathologies of the Attention Economy

I am going to re-read Postman’s “Amusing Ourselves to Death” soon and I plan to write more about what is so strange about the talking heads on TV later. ↩
… except if you are a child. ↩
Source: an interview ↩
Source: A podcast that Arrival’s screenwriter appears on. ↩
How can any respectable news station not use this phrase when talking about an upcoming invasion? ↩

10 Common Features of Communist Regimes

2022-08-27T00:00:00+00:00

The primary source for this list is Dalrymple’s account of his visits to 5 Communist countries as described in his book The Wilder Shores of Marx (Dalrymple). Dalrymple visited these countries in the late 1980s, and several of these countries have since turned to democracy. The 2 other major sources that have influenced me would be movies about life in Eastern Europe during or right after the 2nd World War (such as Pianist (2002)) and Orwell’s dystopian novel 1984. More recent sources include the Vox episode about Cuba released in 2015 and contemporary travel vlogs from North Korea. For lists such as this one, it is hard to pin point where exactly the idea originated from. I would say that the list is an amalgamation of the information provided by the sources I have mentioned and the impression they had on me about the life of ordinary people in the remaining Communist countries of the world. Other media has also influenced me, notably Casey Neistat’s reflections about his trip to Cuba, Conan’s hilarious series of shows from Cuba and the Korean drama Crash Landing on You, in which the protagonist is a North Korean army officer. (An upcoming review of McGregor’s The Party, an account of the governing system in China, will clarify why China did not conform to the typical Communist regime expectations either when Dalrymple visited the other states or today; its curious mix of capitalist economics with Communist ideology has borne the greatest success story in the past half century.)

(I structured this as a list after reading Dynomight’s post supporting lists as a tool for effective communication.)

Rationing of Food Items: All communist countries are forced to ration food items. Why is this? Dalrymple’s dry humor gives us the answer when he is in Vietnam. “After 50 years, he [the leader] realized that the peasant must be paid a fair market price for his harvest. Realizing something that obvious late in life is original.” The “means of production” is owned by the government in communist states. This means that every farm becomes a part of the large government farm. Everything that is grown in these farms is the government’s property by default.¹ The harvest is then split among the people in the country. The key question in this setup of communes is what motivates the farmer to produce a good harvest when they are going to get the same amount no matter what they produce above the official quota? The answer, generally, is nothing. And thus, there is a severe shortage of food produced in the country and an even more severe shortage of food that must be imported from other countries. (Communist countries are almost uniformly shunned by trade partners.)
The Waiter Rules: In a country where food is scarce, the waiter holds as much sway as the bank manager, with the power to sanction a loan, does in capitalist society. Repeatedly, the author runs into disinterested waiters drunk on their power to apportion food among the poor customers at their establishment. (Remember that there is no private property in communism.)
Local Currency is of No Value: As the country produces little, it exports even less. This in turn causes the currency to have no real value at all. When foreigners come into these countries, they are forced to buy the local currency, so that the government can get its hand on foreign currency, which it will use to import the minimum amount of food that is required for the members of the Communist Party. Foreign currency (such as US dollars), cigarettes and cans of gasoline are the usual substitutes.
Megalomaniacs in Power: Dalrymple excels at pointing out why megalomaniacs tend to be in power in a communist country, unless interventions are made specifically to keep them out. The forefathers of communist thought, Marx and Engels, repeatedly said that the human mind is weak and will be attracted towards consumerism and materialism, that communism was the system through which the human mind could be remade into one that appreciated the beauty of enforced equality as against the ugliness of unthinkable inequality, which they predicted (rightly) would be the outcome that capitalism would produce. The leaders of communist rebellions and countries strongly believe that every human who has lived before them has had this weakness and that they are the first ones who were able to overcome it and become better men (after the masters, of course: Marx, Engels, Lenin, Stalin, and to a lesser extent, Mao.²) In order to believe this “primacy of thought” among all humans who have ever lived and to carry out actions that are required to bring this dream of a “new man” to reality, one has to be a megalomaniac, with a huge ego and an even larger portrait of himself at every street corner
Grotesquely Large Roads with No Sign of Life: There must be no private property in communist states. What, then, is the purpose of an 8-lane road? Wouldn’t these roads be used by private cars? Surely, public transport would not require roads this big unless there were to be 8 parallel buses plying between 2 places? Here, Dalrymple begins pointing out the hypocrisy inherent in the minds of the communist leaders of the late 20th century: Their pathway towards development is different; but their idea of the finished product is the same: The acquisition of material goods. By constructing roads and bragging to foreigners that they have such infrastructure or that their roads are never crowded betrays their willingness to give up on their ideology if it is for the (funnily noble) cause of convincing “useful idiots” in foreign countries about their country’s rejection of “backwardness” and its adoption of “modernity.”
1. Dalrymple points out that there is a ban on owning bicycles in North Korea. This ban was lifted in 1992. It seems that there are even bike lanes now.
Intractable Bureaucracies: The Communist state will require you to fill in 50 pages of application forms before you can get your monthly ration. What purpose does this serve? The answer is twofold. First, it keeps you occupied. If you are busy running around getting signatures, filling forms, and understanding procedures, you might not notice that they are meaningless and the forms are most likely incinerated moments after you turn them in. Dalrymple argues that this is essential to a paranoid state because it will restrict the time that ordinary people have to plot a rebellion. Second, it demonstrates the control that the Communist Party has on every day life. Intractable bureaucracies exist in democracies as well, but there, they serve a genuine purpose (the government’s purpose to help its citizens). They become intractable because they are inefficiently run.³
An Irrational Fear of Rebellion: Communist states almost always come up as rebellions. The rebels overthrow a sitting government and establish their new state. This makes them paranoid about the next rebellion; a government that came to power through a rebellion could just as easily be overthrown by another rebellion. The lengths that the different states go to in order to prevent even the rumblings of a rebellion is the scariest feature of implemented Communist thought. In the police states of Eastern Europe, such as Romania, the secret police is everywhere and everyone is an informer to the secret police. There are so many informers that the secret police would see agents reporting on the suspicious activities of other agents, who they think are civilians. In the Communist states in Asia and South America, the secret police is undoubtedly there but takes over less of the daily life of ordinary people. It seemed to me that the Vietnamese people that Dalrymple meets on his travels were far more open about their dislike of the government than those in Europe. In Vox’ video about Cuba’s homegrown network, locals are open about their dislike for the government’s curtailing of the internet.
Production is The End-Goal: If consumption is evil and strengthens the hand of the capitalist, production must not be for the sole aim of consumption. However, production is clearly important and shows the state’s strength and its ability to function. So, production is defined as the end-goal. Production statistics for food, coal, poetry collections, and novels are ceaselessly announced on state television. These statistics are the tool that the government uses against foreigners who might claim to locals that their country is backward. The few occasions that foreigners get to interact with the locals are closely monitored. The “production supremacy”-related propaganda will serve as a safety net if some information about the relative abundance that is the norm in foreign countries happens to get through to the locals. Orwell’s memorable account of the essential nature of war for a Communist nation in his dystopian novel 1984 also comes to mind in this context.⁴
Propaganda: This is not surprising. Even Authoritarian leaders in democracies are weak for propaganda campaigns and never miss the chance to lie about something if they can. Dictators in communist countries are paranoid about a non-state-sponsored narrative about their country’s failures breaking through and fomenting a rebellion; a rebellion is very afraid of another rebellion. Dalrymple, in particular, encounters various kinds of propaganda wherever he goes. Perhaps the worst was the “Juche idea,” the state ideology of North Korea. The author quotes a few paragraphs from the text that was written as the definitive explanation of this idea. It is clear from these few paragraphs that it is meaningless drivel, with absurd repetition of words and phrases and no underlying concept worth conveying.
Single Overarching Narrative: There is no space for confusion in a Communist state. “A thing is either compulsory or prohibited,” Dalrymple says. As he witnesses in Albania, tourists are shepherded from one museum to the next. Each museum essentially tells the same story: a rebel who rallies the country and becomes a charismatic leader in his youth, the revolution, the incredible development that the country sees post-revolution. This narrative does not allow such banal things as the truth to interfere with the fantasy that it is based on. Never mind that most people have to stand in line for an hour to get an ice cream. Never mind that they use cigarettes instead of the local currency. The narrative will always portray the leader as the only possible savior and as the greatest person to have ever lived. This narrative becomes so repetitive and formulaic, that Dalrymple starts making a passing reference to it in his later visits. It makes little difference to him what is being said, and I suspect that the residents of these countries are the same. Vietnam was perhaps the country where this was least obvious. After a long guerrilla war, it was clear that there was no single person or entity to glorify. Also, he makes a good observation: “The falsehoods in propaganda must be proportional to the failures in the real world.” Vietnam won the war; this was a truth which negated the need for any other big lie. For countries like Albania, which played a minor part in World War 2 and the fight against Nazis in Germany, their minor role can only be justified by larger-than-life claims about their pivotal role in World War 2.

If you like to split hairs, then you might point out that the harvest is not the government’s property. In fact, it is the “property of the public.” “Everyone owns everything and the government manages it for them.” ↩
I was surprised by Dalrymple’s account of the leaders in Eastern Europe who openly criticize Mao in their writings. ↩
It is worth noting that in a democracy the citizens never question that the government has absolute control over them; it would be a fool’s errand for the government to go around convincing people of something that they already accept to be fact. ↩
In a brief extract of a book that the main character of 1984 is reading, Orwell writes about why war is essential and to be at war constantly is a core part of the modern state that he depicts in the novel. He says that War is the only “government policy” which can support an unending consumption without creating any progress for normal people. So, to be at war is to keep society stagnant; to leave power structures intact; to conspire to stay in power eternally. ↩

Losing Control of Our Screens and Data

2022-08-23T00:00:00+00:00

After nearly a decade of using the internet daily, it is clear that users have little control over what is displayed on their screen. The control has been ceded to capitalists wishing to sell you something. I am not just talking about the algorithmic social media that is the rage these days. (TL;DR Tik Tok is becoming more popular than Instagram because their algorithm is better and Instagram is getting anxious about it.) I’m talking about advertisements. Newspapers, radio and television have never given the viewer any option about the advertising content that they hear/see. On those mediums, it was easier to distinguish between content and advertising. When the commercial break begins, I can mute the television. With newspapers, I can skim past pages that are advertisements. When an ad is disguised as a search result marked by a greyed out “Sponsored” label which is very small and designed to be hard to notice, the user has no choice but to engage with the ad as if it were a legitimate result.

I have gone to frankly absurd lengths to ensure that advertisements are not seen on my home network. I have installed an ad blocking DNS server, which requires some periodic maintenance. I have VPN setups to ensure that this DNS server can be used even when I am on mobile data; where network providers and the phone software forces you to use the provider’s DNS server. I have installed the uBlock Origin plugin on my computer and I avoid websites which give you some self-righteous message about how their content is paid for using ads and you should whitelist their domain: “Our ads are not ads. They are the latest thingamajig and very secure.”

How did we get here? How could a set of technologies aimed to create a network of computers turn into this?

One obvious (and low-hanging fruit-type) answer is capitalism. Capitalism is the overriding philosophy on today’s internet. Most of the Internet is designed around the facet that it is a conveyance for the age-old salesperson. There is something to be bought and the viewers are going to do the buying. Why are they going to buy anything? Because capitalism connects the society’s value system to things, and the more things you have, the richer you are perceived to be. (Have you not had the experience of scoffing at someone who said they don’t have a Netflix subscription?)

Inversely, if you don’t have things, then you are one of 2 things: poor or minimalist. Poverty has always been the same. Today’s minimalism is the life of the average person from a few years ago; a person who used anything they bought for more than 2 years; someone who liked to buy things instead of subscriptions to things; someone who had all their photographs in a photo album or a 1 TB hard drive, instead of the latest cloud storage offering.

The Best Phone (Only For 2 Years Though)

Apple (famously) spends a huge amount of time and effort on each of their phones. In the bazillion biographies of Steve Jobs, one thing that gets focused on over and over is that the company is detail oriented. Tim Cook, current Apple CEO, is known to negotiate contract prices upto the 6th decimal point because that is how detail oriented they are. What happens to their products that are built with such care and precision? They are chucked out every 2 years.

Every 2 years, Apple releases a new iPhone. A lot of people with the old iPhone walk into an Apple Store and trade their existing iPhone in for the latest one. They get a 50% discount on an insanely overpriced phone. (They also get the social status of owning the latest iPhone. Apple makes a point to make the backs of phones, the part that is most easily visible to everyone except the owner iconic and different on every phone. Other phone makers have caught up to them in this department. The more iconic the back of your phone, the more people will buy it for the social status of owning it.)

What happens to the old iPhone? It is dismantled and used for the parts inside it by Apple. They will use these parts to make the “best phone,” 2 years from now.

Well. Why would a company that is so focused on details and building good products be so insistent on getting its users to keep buying the new iPhone? Money. This is probably banal. Apple wants to make a lot of money? Duh, of course they do. I think this shows the deep-seated contradiction in the philosophy of a company that claims to build the “best” products. If the product is that good, why are you insisting that I change it every 2 years? If the product is that good, why do you stop sending software updates after 5 years? Apple and its attention to detail is superfluous. Even if Apple phones were shoddy products starting this September, people will continue to buy them because it confers social status. The myth of “the Apple product” has been built over 15 years of nearly constant propaganda; this myth precedes the product in the mind of the consumer. Nation states would be put to shame by the overarching ads that Apple releases each year. Memorably satirized by The Onion: “I will basically buy anything if it is shiny and made by Apple.” AirPods Generation 1, anyone?

I am glad to see the latest generation of teens scoff at the origin stories of these large tech giants. Their reaction seems to be, “Google was started in a garage? Umm, okay. Is that supposed to be impressive or something?” and I like it a lot.

They seem to know something that I did not. I was enamored by Google’s ability to go from a small company in a garage to where it is today. I was an ardent follower of the “Steve Jobs myth;” the founding, the firing, the return, the iPod, the laptop comes out of an envelope, the iPod + phone = iPhone talk: Sign me up for all of it! If this ain’t entertainment, I don’t know what is.

I continue to admire Zuckerberg’s meteoric rise, his intention to keep control of a company that has changed culture. (Maybe I should not admire it though, because the change has been pretty bad in every case.)

This phenomenon of products being marketed as amazing and paradigm-changing when they are launched and becoming outdated and boring a year or two later is everywhere. This is true with the ultra fast-fashion brand Shein. On Shein, consumers buy clothes as fast as Shein can produce them or as fast as they go viral on TikTok. These two businesses tie into each other. Consumer choices are now controlled by how fast trends change on an online platform that is known for dizzyingly fast trend cycles.

Subscriptions for Everything

There are subscriptions for movies, music, sports, gyms, and cosmetics now. But it doesn’t stop there. In each of these cases, perhaps you are paying for a service or a good that is not already in the hardware that is “yours.”

New versions of old software are trying to convince you that they are somehow completely different from whatever came before them. Microsoft Office is a good example of this. For decades now, Office has remained basically the same and I am grateful it exists because it has simplified computing for a lot of people. There have been a bunch of new features which power-users use, but if you are using Office to write something, print it, and then have people sign on it, the product has remained unchanged.¹ Even Office is now different. Instead of selling a software license, Office now wants you to buy Office 360, the Google Docs clone from Microsoft. Why? Weren’t most users of Office happy to write things into Word files and then send them around on old-fashioned email? (Email is one of the last surviving inter-operable communication technologies.)

The subscription model has caught on like the last raging wildfire somewhere in the world. BMW and Mercedes are now selling subscriptions for hardware that is already inside their car. Everyone wants you to buy a subscription.

I recently found out that PocketCasts introduced a feature which enables users to put podcasts into folders and categorize them. This feature is available only to premium users. And how much does Premium cost? $1 a month. Does this make any sense? I don’t get it. Why can I not just purchase the Folders feature for life? What is the service that PocketCasts is selling when I buy the Folders feature? I like PocketCasts and I recommend it to everyone constantly. But seriously, what is the feature that I am subscribing to?

Goodbye, Data

Data subscriptions were an amazing find for Google. Google Drive sells storage in the cloud. You can upload a lot of data and forget what you uploaded. When you “run out of space,” Google will prompt you to pay for more (as a subscription, of course) and you will have little choice because the initial 15 GB of data that Google gives you is so much that you would have used it with abandon and you no longer know what is in Google Drive, Dropbox, and One Drive.

If you are unwittingly using Google Photos, then your photos are being backed up all the time. If you set up a new Android phone recently, the default is to “backup” all your photos to Google Photos.

Of course, all of this is designed to make moving out of Google incredibly hard. Here is a hypothetical scenario:

If Google were to decide that they are going to stop iPhone users from uploading photos to Google Photos, how many users would switch to Android in the next few days? I would guess that a lot would. Using iCloud instead is possible, but to get the advantages of facial recognition on Google Photos, you would have to continue using it and not switch products.² If Google is allowed to lock out iPhone users, they would almost certainly be able to prevent users from exporting their data.
Governments have “anti-trust” proceedings which can probably stop Google from doing this. But Google has a distinct advantage. There are about 200 countries in the world, and there would have to be suits in every single country against Google. In some countries, the legal process might be quick and Google might be forced to comply within a few weeks. In other countries, legal proceedings will drag on and on for at least a decade. While people wait for the “landmark anti-trust ruling,” they will buy Android phones and send even more data Google’s way.

I don’t think that any of this will happen. We live in the age where a change is defined as good or bad by the outrage that it generates. So, Google will not want to do this. However, a user of Google Photos will almost certainly consider good integration with Google Photos a major feature of any future hardware that they buy. If a Linux / Windows / Apple computer can not integrate as seamlessly with Google Photos as a Chromebook can, they will go for the Chromebook.

This kind of scenario is playing out in a few different arenas; not just cloud storage of photos.

The example that is closest to my heart is Kindle and Amazon. Amazon is known for large fundamental changes. Kindle is a one-stop shop for nearly all books right now. I have a library inside of Kindle, with books that I have already paid for. If Amazon were to stop sending software updates to my Kindle and then they claimed to not support my Kindle version for books that were published after today, then I would be forced to buy the latest Kindle.³ Already, there is the “supported devices” list on Amazon’s website. From an engineering standpoint, it should take little effort to prevent anything published after a given date from being delivered to any device that is older than a given version. This decision might not even be made in bad faith; it might simply be because newly published books have some cool feature and the effort required to make the book backward compatible is not justifiable because the number of people using these old devices is few.

There are more and more real-life scenarios which will affect physical things, and not just the connecting tissue between software and hardware like our screens. Internet of Things devices like Google Home are configured to integrate with the lights in a house and its security system. If the security system does not integrate with Amazon Alexa, tough luck. You will just have to stick with whatever you already have.

Personally, I want to stay away from these kind of services. But I recognize that I will be in the minority, as I have started feeling in the smart speaker boom. When I was buying home appliances, the campaign at the electronics store gave away Google Home smart speakers to anyone who wanted it and was buying something else. It is hard to ask people to refrain from taking something that is being given to them for free. Certainly, high-minded arguments about privacy will not work. The simpler economic argument that there is no “free lunch” is slightly more effective.

If the metaverse is where the entertainment of the future is going to come from, people will have little choice or say in the matter. Much as the television and the Internet changed society without a vote, the present status-quo will change and future technologies will all appear without any debate. The lack of technological progress is only going to be seen as a failure.

We don’t need and should not use “upload and forget” one-click solutions. We need tools that whose operation is familiar and configurable:

What we need … is tools–not colleagues or friends. And yet technology companies learned long ago that their products are far more appealing–and more profitable–when humans manage to bond with them. As robots increasingly come to replace humans in retail and food services, these humanoid touches begin to seem especially sinister. Walmart, one of the companies that recently began using robots in its stores, is already implementing training programs to help their employees transition into other sectors, knowing that the number of retail positions will soon decline as machines take over.

– p.112, God, Human, Animal, Machine (O’Gieblyn, Meghan)

Some might argue that the product has actually become less easy to use in the past few years. ↩
I had the chance to see this feature in action and I am stunned by how good it is at identifying pictures of babies and matching them with adults. When it worked with nearly 5 years of photographs, I could not identify a single mistake in its recognition. This feature really made me feel like I was missing out on something big by not using Google Photos. I can see the power of a Photos product which can identify people and tell you about them exactly when you are thinking about them. ↩
I maintain a separate list of books locally. And I export all my notes to CSV files and store them myself. So, I am not worried about losing the notes or the library’s list of books. But I am worried about losing the books themselves. ↩

The Keyboard Logs

2022-08-06T00:00:00+00:00

I have been experimenting with keyboards for the past several years and in February this year, I switched to a keyboard which I hope will be the last keyboard I ever buy. The whole point of switching keyboards all this while was to buy the right version of this keyboard, the Ergodox EZ. It is a split keyboard with an ortholinear layout. It is an expensive keyboard and can be customized to an (almost) limitless extent. It has several extra keys which can be programmed to do whatever you want them to such as function keys or macros. This post is a log of the keyboards I bought (and sold off) during this process.

Basics

There are some common terms which are used constantly in the discussion of keyboards, and mechanical keyboards in particular. I don’t want to re-explain these topics here. I will link to short explainers about each term:

Goal

I wanted to pick the right type of key switches for this keyboard. This keyboard can be customized to do a lot of things, but the key switches inside it can not be changed inexpensively. So, I wanted to get this key switch choice right.

The keyboard uses the “split” form factor with an ortholinear layout. I wanted to find out how much the split layout improved my posture and comfort.

Finally, the ortholinear layout itself. I wanted to understand whether having the keys directly above and below each other, instead of the typical staggered layout is helpful at all. Switching from the staggered layout to the ortholinear layout is a large investment as it requires one to unlearn years of muscle memory using standard staggered layout keyboards.

With these goals in mind, I started searching for similar keyboards which were cheaper and could give me a feel for what using the Ergodox would really be like. I found out about the Ergodox EZ some time in 2020 when everyone started working from home and started talking more about their work setup. My keyboard journey starts well before that.

First Keyboard (2014-2016)

August 14, 2014: Logitech MK270r Wireless Combo Keyboard (Membrane)¹

Looking around for a keyboard in my second year of college, I saw that the Logitech keyboard combo was widely used by my peers. (I had spent the first year of college using my laptop’s in-built keyboard. I had stayed away from my laptop for the most part of my freshman year anyway, so the keyboard did not matter as much.) This combo connected to the computer using a LogiTech USB dongle. I saw it around almost constantly, and I felt better investing in a brand which I knew about already.

This was a membrane keyboard. This means that there is a single membrane between the switches. When a key is pressed, it touches an electrical contact on the membrane, which in turn touches the underlying circuit board and transmits the required signal. All the keys are working on the same membrane and bending it when they are pressed. This means that the actuation force (the force required to send the signal) is variable. It is also higher than mechanical keyboards (The membrane’s resistance is quite high.) The force required to actuate each key is variable and depends on the quality of membrane. But the actuation force varies between 65 and 75 grams, with cheaper keyboards requiring more force to actuate the keys.

The actuation force is the force that you must apply on each key when you are pressing it as you type. If you are typing a typical e-mail or a Slack message, it might be about a 100 words, which might be about 400-500 characters. This would mean that you have to apply about 65 grams of force 500 times, or 32.5 kg of force for a single message. This number seems quite high. When I was not typing too much with my membrane keyboard, I did not have any pain in my wrist or my fingers. My setup in college was highly un-ergonomic and so I could not really blame my keyboard for any other pain that I did have. (I did not use an office chair and I was mostly sitting without any back rest.)

Nevertheless, towards the beginning of 2016, as I finished my third-year in college and entered my fourth year, I started feeling the stress in my fingers. In my third-year, I prepared for interviews for landing internships and worked at my first remote internship. During this time, I noticed that the membrane keyboard was really taking a toll on my hand. Every time I would stop typing or take a break, I could clearly see that my hands were fatigued and I had to work through the pain for a period of time. This intense period of typing during the first half of 2016 convinced me that I needed a better setup and I started searching for this setup.

Getting On The Mechanical Keyboard Train (2016-2018)

July 14, 2016: TVS-e Bharat Gold PS2 Wired Keyboard (Mechanical) (Cherry MX Blue)

Mechanical keyboards seemed to be the universal answer to pain in the fingers and wrist induced by typing. As I started looking for the models that were available in India, I started realizing that all of them were expensive. In fact, I could not find anything which was cheaper than INR 10,000 during the time. I was in college and I did not want to spend so much money on a keyboard which may or may not solve my pain problems. So, I kept looking. About a month after starting my search, I ran into this TVS Electronics keyboard on Amazon. It seemed to be too cheap to be real. At INR 2000, it was far cheaper than anything else I had seen on Amazon or on YouTube. (The Indian tech YouTuber scene in 2016 was not as good as it is now, so I was able to find reviews and comparison videos only from American YouTubers, where mechanical keyboards had been popular for a few years.)

The brand TVS is well-known inside India for making automobiles, but I have never heard about TVS manufacturing any computer peripherals. So, I was suspicious of this product. There was a website for the product which described it well. The version that was available on Amazon used the PS/2 port, a port designed 30 years ago, in 1987! Keyboards had moved on to USB a few years ago and to Bluetooth in the couple years before I purchased this keyboard. So, the technology was quite old. I could not find many reviews on the Internet for this product.

Nevertheless, it seemed to be my only bet and after confirming that I would be able to return the product, I ordered it off of Amazon. After using it for a couple days, I was convinced that the product was legit and actually did use real Cherry MX Blue keys. In fact, the keyboard was extremely good and I started using it exclusively immediately after purchasing it. I used it throughout my pre-final and final year when I had to type a lot. I typed both my Bachelors and Masters thesis on this keyboard. I also used it for all of my college placement preparation: writing CVs, practicing competitive coding questions on CodeChef, notes about my placement experience, a large part of this block.

For nearly two years until my graduation in May 2018, I used this keyboard everyday. I was living alone in a hostel room for the whole of this period, so the sound that the Cherry MX Blue switches made was not a problem. I did not move around during this period either (Indeed, there was very little travel.) So, the weight of the keyboard was not a problem either. This keyboard remains one of my favorite keyboards to use. The product I bought in 2016 still remains in heavy use in my father’s home office.

Moving to Japan (2018-2020)

October 24, 2018: Gamdias Hermes E2 7 color (Mechanical) (Cherry MX Blue)

As good as the TVS-e keyboard is, the one thing it is not is portable. Owing to the danger of bringing it all the way from India to Japan and damaging it or not finding a use for it, I left it at my parents’ house before moving to Japan. I intended to buy a Cherry MX Blue keyboard after arriving in Japan.

On my first payday in Japan, I went to an electronics store and bought this keyboard. It was a Cherry MX Blue keyboard, which I managed to figure out using the translator application on my phone and repeated confirmation with the store clerk. My only real consideration when buying the keyboard was budget. I did not know the Gamdias brand very well, and I had not read many reviews about this keyboard either. I decided in the store after comparing some keyboards and their prices. The other keyboards were mostly targeted at the gaming audience and seemed to have too many backlights, that could not be disabled. This keyboard also had an array of backlights, but they could be turned off easily and this was the reason I chose it over the other options on display.

One thing about this keyboard was that it had the JIS layout (and not the typical US layout). The JIS layout is mostly similar to the US layout but there are some extra keys which are used only when typing Japanese. Through some software tweaks on Linux and Mac OS, I was able to use this keyboard effectively at home. Throughout 2018 and 2019, I was going to my office daily for work and I was unable to use this keyboard there because it was too loud. This keyboard was pretty good for typing at home, where I did a minority of my typing. Using this keyboard, for the first time, I realized that all keyboards which use the Cherry MX keys are not created equal. Indeed, a mechanical keyboard is made up of several parts and the keyswitch is a major one, but not the only factor to consider when assessing the quality of a completed product.

I used a (terrible) Apple keyboard at work, where I was typing a lot. That keyboard definitely gave me some finger pain and I wanted to move away from that at work.

I sold this keyboard on Mercari² 3 years after I bought it for little lower than the price at which I had bought it. (I am an employee at Mercari. Future references to Mercari will include a footnote stating this.)

Experimenting With The Split Form Factor (2019-2020)

October 26, 2019: Kinesis Freestyle 2 for Mac (Membrane)

I wanted to experiment with the split form factor. The idea with the split form factor is that the keyboard does not have to be a single unit. It can be split into two pieces, which can be placed at distance. This ensures that the hands don’t have to be squished close together into the typist’s body.

I was constrained to using a membrane keyboard because I wanted to use the split form factor at work, where I was doing a majority of my typing. (I had not started taking Smart Notes yet in 2019.) There weren’t many membrane keyboards which had the split form factor. In fact, there were only a couple that I can remember. Out of these, the Kinesis had good reviews. So, I bought a second-hand one off of Mercari², a marketplace app for second-hand goods. The price was good and there was little risk in this purchase because if I did not like the keyboard I could simply sell off the keyboard on Mercari once again, at a slightly lower price.

I used this keyboard at home for a couple weeks to become familiar with the layout. Getting used to the split form factor was hard at first. The temptation to reach for H or G on the wrong half was strong. (I do not do “perfect” touch typing, where typists use only the assigned fingers for each key. I would often “cheat” by using the left index finger for H or the right index finger for G. This kind of cheating is easily forgiven on a non-split form factor because the keys are close to each other; but it is unforgivable on a split keyboard, as it slows you down.)

In a month or so of usage, I was able to type on this keyboard at the same speed as I used to type on previous form factors. However, there was a major flaw with the design of this keyboard: Although it was using the split form factor, it did not change anything from the US keyboard layout. So, all the modifier keys were on the extreme left of the keyboard (such as Tab, often used in Ctrl+Tab and Alt+Tab and Esc, mapped to Caps). The finger extension for my left pinky finger to reach these keys was very large and strained my fingers. I had to take my left hand off the home-row when I needed to use a modifier key. I had to take my right hand off when I wanted to use the mouse. This flaw made the experience of using this keyboard beyond 2 months quite painful. I went back and forth with this keyboard between my office and home for a few weeks each time; hoping against hope that I would get used to the layout and perhaps even get over the pain. Neither of this happened, and typing on it remained a painful experience.

I ended up selling this keyboard on Mercari². Things changed quite a bit in terms of my work environment in the beginning of 2020.

Trying The Cherry MX Brown Switches (2020-2022)

April 2, 2020: Filco Minila Air (Mechanical) (Cherry MX Brown)

In the second week of February 2020, everyone in my office was sent home to continue working from home full time. I was able to use my Cherry MX Blue switches constantly now, and this was quite a good experience. But the loudness of the Cherry MX Blue switches created problems almost immediately, even though there was no on else in my home.

Namely, I was unable to type during meetings. As it had become commonplace to have a Google Doc opened and present it during a meeting, where everyone could take notes, this meant that I would not be able to type unless I muted my own mic. So, I would not be able to type and speak at the same time. This was a common pattern of my work back then, especially when I was brainstorming an idea or a calculation with a coworker. I realized that the Cherry MX Blue switches were so loud that I could not use them for work.

Google Meet’s technology was still being improved and it did not have the “background noise cancellation” feature that it does in 2022. Even still, it seems unlikely that I would have been able to get away with typing on a Cherry MX Blue switch during a meeting. Searching for a silent version of the mechanical keyboard that I was using, I found the Red and Brown variants. I had heard a about the Cherry MX Red and the Cherry MX Brown switches.

MX Red switches are linear and do not have any tactile feedback. This was a no-go for me; half the reason I was using a mechanical keyboard in the first place was the tactile feedback that I was getting when I pressed a key. This tactile feedback let me know that I did not need to press any longer. It had become a subconscious part of my typing experience and I did not think about it at all anymore. Typing without the tactile feedback, even on a linear switch, would be hard, I felt. The reason you need to apply a smaller actuation force on a mechanical keyboard is because you get feedback when the key has been typed. If you take away this feedback, you are left guessing or waiting for the screen to update you about whether you typed a key. This guessing process always ends with typing more forcefully and getting it right all the time, rather than typing less forcefully and not actuating the key sometimes.

Looking around for a Cherry MX Brown switch keyboard, I saw several large keyboards. The Filco Minila Air keyboard is one that I saw at an electronics store and its small design intrigued me. Did I really need the number pad? I used to use the number pad back then, but it involved taking my hands away from the home row. I wanted to be a proper typist, if at all possible. Getting rid of the number pad seemed like a step in the right direction. The Minila Air had received good reviews and seemed like a solid keyboard from a well-known manufacturer. So, I bought it and started using it daily for work from home.

It solved my “can not type during meetings” problem. The MX Brown switches on the Filco were silent enough that typing lightly during a meeting would go unnoticed. Even though it took me a few weeks of practice before I could really type lightly enough reliably, I could see that the MX Browns were significantly more silent compared to the Clicky MX Blues, which are loud enough to rouse people in deep sleep with the characteristically sharp click sound.

The MX Brown switches on the Filco also felt better than the MX Blue switches on the cheaper Gamdias that I was using before. Again, I felt that the switches were not the sole component that mattered; the build of the keyboard and the manufacturer’s experience also mattered a considerable amount.

An added advantage with the Filco Minila Air is its portability. When I went back to India to work for 3 months from there, I took it with me from Japan to India. I also brought it back to Japan. I brought the keyboard in my cabin baggage. I was afraid it would be broken in transit if I put it in my checked-in baggage. This meant that I had to unpack the keyboard and keep it alongside my 2 laptops on 5 different occasions during the trip, 1 time in Japan and 4 times in India. The experience of doing this in India was harrowing, to say the least. (The severely limited supply of trays in airport security check queues is a genuine crisis!)

After returning to Japan, I knew that I was ready for the final step in my keyboard journey: An ErgoDox EZ purchase with the Cherry MX Brown keys. Before going to that step though, I will digress slightly and explain my brief experience in the build-your-own keyboard world in Japan.

Sidetrack: Build-Your-Own Keyboard Misadventure (March-May 2021)

March 20, 2021: Custom keyboard number pad kit

The build-your-own keyboard world in Japan is extensive. I have a few coworkers who have “built” their own keyboard. The process involves buying a circuit board, a microcontroller, the key switches, and the key caps which will be fitted on top of the switches. Then, soldering all the components together yourself.

I bought a small keyboard kit which, when assembled, would be a 10-key number pad. I don’t have any use for a number pad, but I wanted a kit on which I could test whether I would be able to put together a real keyboard.

This went poorly. I had greatly overestimated my soldering skills (I thought I was “OK,” it turned out that I was actually “terrible.”) And I had greatly underestimated the complexity of working with hardware. In particular, the point at which I realized that soldering was not my cup of tea was when I started soldering the microcontroller. The microcontroller has 12 pins and these are small and extremely close to one another. So, one has to solder everything perfectly and ensure that none of the soldered points are connected to each other, which would be a short circuit and would hamper the microcontroller’s functioning significantly.

After messing it up and trying to fix it, I decided to give up. I was left with the kit and some soldering equipment which I had no use for anymore. The only thing I did learn from this process was how keyboards look like under-the-keyswitches and the effort that goes into making a single keyboard when they are made manually.

Buying The Ergodox EZ (Jan 2022)

January 22, 2022: Ergodox EZ (Mechanical) (Cherry MX Brown)

So, here we are. About 7 years after starting this journey, I have finally reached a keyboard that is sufficiently customizable and ergonomic for sustained daily use. I am continuing with the QWERTY layout, despite evidence that proves that this layout is not very good and involves too much finger travel. The keyboard version that I have bought does not have the names of keys printed anyway, so the switch of the keyboard layout to something modern such as Colemak DH might be in the cards for me. However, I am not going to spend time on it now.

I have spent a considerable amount of time getting my tooling to its current shape. There have been misadventures along the way. They did not cost me much money, but they did take up some valuable time that I could have saved.

Nevertheless, I am glad to have started using mechanical keyboards and I can wholeheartedly recommend the Ergodox EZ keyboard to anyone who knows which switch they want to use.

All the dates in this post are accurate and from receipts of online stores. I went back into my email to dig these out and that was quite fun. ↩
I am an employee at Mercari ↩ ↩² ↩³

Book Review: Global Economic History (Allen)

2022-07-29T00:00:00+00:00

Allen’s Global Economic History: A Very Short Introduction is a 200-page masterpiece. It packs a huge amount of history. It looks far enough into the past to set the scene for the Industrial Revolution and the economic gains seen as a result of it. Then, Allen retreats into the fundamentals of economic development and goes over the main reasons some countries are perennially stuck in the “middle income trap.” He has some advice for such economies and what they might be able to do to get out of this trap. This book is solidly based in data. But Allen does not prioritize data over the story. Economics is the story of real people who live in each of these countries. Allen never loses sight of this, and always puts large economic shifts in context by stressing on how it impacts everyday life. This emphasis on personal experience makes the book a study in the government policies that work and the pitfalls to keep in mind.

The topics is quite deep and consequently, this post is long. I have categorized it into multiple sections for the reader’s convenience.

Introduction: I start with an introduction of the world economy and the large shifts that have been seen over the past few centuries
Standard model for economic development and the industrial revolution in Britain: In modern times, this remains the largest fundamental change in the way people worked and the way their lives were organized, with people moving from villages to cities and leaving farming for industries
4 topics related to middle-income economies
- The middle-income trap
- Why literacy must be perceived as useful for people to send their children to school
- How imported technologies and their adaptation is essential for economic development
- The big-push industrialization model for economic development
Africa’s woes: Finally, I talk (briefly) about the African region’s woes and some reasons for the lack of development in that region since 1820, and why it remains the poorest region on the planet.

Introduction

The economic history of the whole world is a nearly intractable subject. Allen begins with a table that shows the difference between the present and the past.

Region	GDP per capita in 1820¹	GDP per capita in 2008¹	Multiplier
UK	1,706	23,742	13.91
US	1,202	30,152	25.08
Japan	669	22,816	34.10
World	666	7,614	11.43
China	600	6,725	11.20
India	533	2,698	5.061
Sub-Saharan Africa	415	1,387	3.34

The countries that were rich in 1820 continue to be the richest of the lot. Nothing surprising there, those countries had a head start. Sub-Saharan Africa is the region that has grown the least in the intervening 2 centuries. But the numbers look encouraging, right? Allen dives into the details of what these numbers mean and what one should take away from the development paths that these economies have taken.

Over the past few centuries, the trajectory is easier to visualize. Between 1500 and 1800, European powers discovered many regions on the planet and colonized those regions. Most of the trade in the world was done by colonial powers who imported products produced in their colonies. In 1750, China and India dominated world production with a 33% and 24% share of the total production respectively.

After industrialization kicked off in Britain, products were produced in Britain and exported to the world. These products were much cheaper than those manufactured in the colonies (such as cloth from China and India compared to the cloth made in Britain, using the cotton mill.) As the cheaper products took over market share from China and India, Britain’s economy strengthened at the cost of China and India’s development. Until the 1990s, their share of global production kept declining.

The colonisers had another advantage: Not only did they produce cheap goods using modern technology, they also controlled the prices of competing goods. So, Britain could take a commodity like salt, which can be cheaply produced in India, and use import tariffs to ensure that salt imported from Britain is cheaper than salt produced in India. This diabolical use of colonial power was the reason for the “head-start” which colonial powers gained. They were able to destroy their competition and they could compel their customers to buy from them.

In India, after independence, from 1947 until Liberalization efforts in 1991, the economy struggled to grow. China was stuck under Mao’s disastrous implementation of communism which lead to the death of millions. Starting in the late 1980s, China started catching up very quickly. Indeed, most of the world’s improvement in the 4 decades between 1980 and 2020 can be attributed only to China’s growth. India was never able to capitalize on this trend of East Asia’s economic strengthening in the 21st Century, and thus, was never able to recover from the colonial collapse.

Looking into the future, it is clear that China will soon overthrow the US as the largest economy in the world. When that happens, the world would have come full-circle. (India would have missed the bus to make this cycle work and will be stuck in the middle income trap.)

Why did the Industrial Revolution happen in Britain, and not in India or China or Germany? This is the key question of economic development. Allen tries his best to convince us that geography and access to resources was the driving force of the answer.

Standard Model of Economic Development

The standard model of economic development was followed by US, UK, Japan, and partly, by the USSR. The idea is that a country can not advance itself by only exporting surplus agricultural produce, so it should take steps to ensure that it has comparative advantage on some products. It should import what can not be produced cheaply locally and it should export what can not be produced at a lower cost elsewhere. To produce more and more things locally and at a discount to its global rate and shipping cost, it should import technologies from more advanced economies and protect businesses in their early stage until these businesses can learn to innovate on the imported technology and improve them.

To go from being a primarily agrarian economy to a primarily industrial one, countries will generally start with cheap capital. There is a need for a banking system which will invest in development inside the country.
This capital is then used to build the requisite infrastructure around the country: roads and railways. During the building of this essential infrastructure, the country will develop local industries for manufacturing of materials such as steel and the operational knowledge for running railways, etc.
As the infrastructure is coming up, countries will incentivize the creation of new businesses and the manufacturing of more and more things domestically. During this period, the products that are manufactured domestically will be inferior and expensive compared to the “state-of-the-art” elsewhere. However, governments must implement import tariffs to ensure that imported goods do not compete with domestically produced goods. The only way to foster industry in the early days is by protecting it from competition.
As industries inside the country find their footing with manufacturing and are able to import technologies and adapt them to local settings, they will figure out cheaper ways to produce what was being imported from other countries. This will increase the competitiveness of their products abroad and the economy will enter a virtuous cycle of innovation and competitive production.

This standard model worked very well for Britain, because it had cheap capital and cheap coal.

The model worked for the rest of Northern Europe as well. Their proximity to Britain and their colonies bolstered their ability to catch up to the fast industrializing Britain. Some European economies were even able to overtake the British economy in the industrialization race.

It worked very well for the other independent economies that were trying to develop during the time of Empires: The US was able to keep British goods out by implementing tariffs. The US also had practically free land, as the British colonies on the East Coast started expanding into the Wild West.

It worked well for Japan: Japan had cheap capital and the ability to adapt technologies which were imported from the West to fit local restrictions.

However, this model did not work for the colonies. As it is, colonizers do not want colonies to develop too much. Colonizers will produce many things cheaply in their own countries and start pushing them into the colony. This prevents businesses in the colony (if these are allowed to exist) from ever having the protective environment that is required to start producing.

In the case of building infrastructure, again, the colony is seen merely as a market for the colonizer’s goods and nothing is manufactured locally. In The Great Hedge of India (Moxham), Moxham demonstrates how the British deviously used tariffs to ensure that salt produced in India would be sold at the same rate as salt imported from Britain. When railroads were laid in India, nearly 61,000 km of railway tracks were laid by the British using British steel. India was never able to develop her muscles in the large-scale manufacturing industry, which is a requisite for entering the next stage of economic development. This is the reason the standard model never worked for colonies and they had to start from scratch when they became independent.

Things got much worse after World War 2. The standard model stopped working. The global banking system had become too interconnected for capital to be invested in new economies and be cheaper relative to the cheap labor that was available in these economies. Some misguided protectionism after the war lead to stagnation and an utter lack of accumulation of knowledge or skills in the developing economies. When the economy was opened up after this period of protectionism, countries realized that they were forced to import most products that they needed due to the principle of comparative advantage. All of this culminated in the stagnation that Allen calls the “Middle Income Trap.”

Large institutions like the IMF, World Bank, European Union have made institutions in African countries which have not paid off because these institutions want to push the “free market” idea into small, maturing economies. Only large economies such as India and China were able to ignore World Bank advice, while economies like Pakistan and Sri Lanka had to follow this advice, and now, their economies are on the brink of collapse. This is covered in Reinhart’s “This Time Is Different.”

The standard model also relies on the Minimum Efficient Size (MES) of innovations. For every innovation, it is worth investing in the innovation only if the factory that uses it produces at the MES. When Latin America tried to apply the standard model in 1980, the MES of state-of-the-art innovations had skyrocketed. A single factory of MES size would be enough to serve all the domestic demand. This stifled the need for innovation and a monopoly was an efficient solution in this situation. But a monopoly does not foster innovation or competition, and is bad for a country’s economy.

Industrial Revolution

The Industrial revolution in Britain was a key turning point for the West. They had already gone from being mere monarchies to colonizers; but industrialization gave them the incentive to double down on colonization and squeeze colonies of any chance of development. Cheap coal, cheap capital, and high wages came together to make Britain the center of the Industrial Revolution.

The prohibitively high cost of labor and the relatively cheap cost of capital pushed research forward in several domains and Britain ended up improving the production process of many goods. The leading producers of these goods were often countries that Britain had already colonized, so they were able to squeeze the colonies from both ends: reducing production in the colonies through British brawn, while improving production processes at home through British brain.

Allen goes through an impressive list of innovations which all came out of Britain during this period. While the list may be impressive in itself, it should always be kept in mind that colonization and the attendant atrocities were the prerequisites for these innovations.

Cotton: Chinese and Indian cotton was surprisingly competitive in Britain, even as late as the 1770s. And in West Africa, British cotton was far too expensive. The spinning jenny and the mule made sense in Britain’s high wage environment. They gave a total return of 40% when employed in Britain. Whereas in France, the return was 9% and in India, it was a meager 1%. So, India saw no mechanization of cotton spinning. As the installation of these machines kick-started future improvements in cotton spinning, the setback faced in this period was of colossal proportions for India and other countries where labor was cheap.
Power Looms: Wages in America had surpassed Britain in the early 1830s. So, as expected, Power looms were embraced with much more gusto in America than across the Atlantic.
Steam Engine: The initial coal requirements for introducing a steam engine were very high. In the cheap coal environment in Britain, this was not a problem at all. Research continued steadily and significant improvements were made. In 1730, Newcomen engines required 20 kgs of coal² to generate 1 horsepower of energy. By the late 1800s, this number had gone down by a factor of 40 to 0.5 kgs of coal.³ British innovators cut through the technological frontier because they were the only ones who had access to the high initial requirements of coal. If countries could not clear this initial barrier, they would be forced to import technology from Britain at a premium. (This is what Japan did when railways were introduced in
1. British engineers are credited in Japanese railway museums with installing the first tracks
and trains.)
Railways: Starting in 1830, the power of Steam engines was successfully applied to locomotives which ran on iron rails. The network grew around Britain very quickly, increasing to 10,000 km in 1850 and to 25,000 km in 1880. The locomotive was popular among passengers who wanted to avoid bad, unpaved roads for long-distance travel. It was also a popular mode of transport for general freight.
Ships: The Steam engine transformed shipping. The sail lost ground and steam took over, with ships carrying the coal required to power their Steam engines. Initially, the space required to store the coal required for a cross-Atlantic journey was too high. However, as the weight of coal required per horsepower plummeted, the steam engine became the most common way for powering Atlantic crossings.

While England had the first-mover advantage in the Industrial Revolution, the rest of the continent caught up to Britain by the 1870s. Indeed, Germany and France had surpassed England in cheap steel production.

Strikingly, during this furious period of innovation between 1770 and 1870, there were no major inventions anywhere else in the world.

Escaping the Middle Income Trap

Some countries are stuck in the “middle income trap.” A part of their population is always in abject poverty and the doors of social mobility are permanently closed to them. Why do some countries get stuck in this trap?

Every country starts out with no real technology. They use the tools that have been used since antiquity for the first part of their development. During this period, wages are low and businesses have no incentive to invest in technology which will increase the productivity of each worker.

As the productivity does not rise, the total amount of product produced by the country does not rise either⁴; and the country is stuck in a trap. There is no “natural” way for an economy to get out of this trap. The market is running efficiently by not innovating. There must be an external stimulus to go from the “Vicious Cycle” to the “Virtuous Cycle” in the diagram above.

There are a few different paths that Allen describes. For each path, Allen describes a case where it was successful in the past.

The first strategy is the most common method to get out of this trap. Some countries increase workers’ wages. This creates incentive for business leaders to squeeze more product out of the workers’ (now limited) time at work. This incentive will turn into innovative technologies which are either invented or imported. This worked well in England where machines to spin raw cotton into yarn kick started Industrialization.
The second strategy is government investment. The government invests a huge amount of money in businesses and forces them to modernize their infrastructure. This strategy was most effectively used in recent years by China. The question remains, where should this money come from? For rich European countries, this meant digging into their, coffers which had gotten rich through the exploitation of their colonies. For poor countries or countries which did not colonize anyone, it meant taking loans from foreign investors. There is a pitfall here though. When a country is large enough and the central government is powerful enough to stamp out (unreasonable amounts of) corruption and internal strife, foreign investors never gain enough power to exploit the country. However, if the country is corruptible or its economy too small to make much of a dent, it is ripe for exploitation by its debtors. Foreign debtors are greedy and can often push the country into unwanted spirals of debt restructuring to the debtors’ benefit. So, being in debt can be a dangerous strategy for some. The infuriating story of France’ exploitative arrangements with Haiti in order to collect the independence ~~debt~~ ransom is a classic example of an economy which was exploited by its debtors.
The final strategy is brute-force improvement of productivity through increased work hours and technological adaption. The adaption of imported technologies is important for economic growth and there are many instances in the past where it is clear that countries that can adapt imported technologies fair better than those which import technologies and try to use them as is, without redesigning them for local economic conditions. Asian economies which flourished and reached the top of the world economy after World War 2 are prime examples of this technique: Japan, South Korea, and Taiwan.

Incentives to Become Literate

Literacy is good. It informs the population, enables them to choose better leaders (until recently, at least), and increases the opportunities that ordinary citizens get. In the TV show Downton Abbey, an undercook learns mathematics and history from a local school teacher. Soon after, we see that her confidence increases and she feels comfortable saying out loud that she does not have to be an undercook all her life. This is the character of social mobility: If you want to convince a majority of your population that becoming literate is good for them, then they must see the benefits first hand.

Northern Europe saw significant increases of literacy by the 19th century. Most of these increases were powered by the imperialism that had become essential for most European nations. As imperialism advanced, these countries became rich and products that were imported from China were soon produced cheaply at home due to industrialization. The demand for these cheap products went up and wages increased as people saw that there was use for their money.

As wages increased, parents sent their children to school and children studied as they could clearly see that educated people had better opportunities in a fast industrializing world.

Country	Literacy in 1500	Literacy in 1800
England	6%	53%
Germany	6%	35%
France	7%	37%
Italy	9%	22%
Spain	9%	20%

Efforts were taken to keep children in school because the economy demanded literate participants. For e.g., in the American settlement, literacy was a considerable advantage after America’s independence in 1776. State schooling and mandatory attendance were introduced in America almost immediately after independence, and trade with Europe continued unhindered. Children went to school and literacy spread because it was economically advantageous. On the other hand, in Mexico, only 20% of the population, almost all of the White men, were literate. The remaining population, mostly non-White peasants, did not see any economic advantage to becoming literate. This held them back. While the highly literate men in America had access to journals and magazines about increasing crop yield, the peasants in Mexico slogged away using methods that had been used by their forefathers. (Due to the nature of colonization, even if there had been a demand for universal education, it would most certainly not have been provided by the ruling class.)

The main takeaway for me was that it is important to *convince the population that the benefits that literacy will accrue to the children far outweigh the costs to parents and the time spent by children in schools.* Parents also need to be convinced that the sacrifice of free labor on the family farm is worth it. This fundamental equation has been forgotten in some countries. Even in India, where the government claims 100% literacy using a low bar for literacy (being able to sign your own name), the government’s willingness to bolster the education system has been half-hearted. At several points during the Covid19 pandemic, schools were closed while malls, restaurants and factories were open. Most schools in India are not air conditioned and they are well ventilated through open windows. Given the pandemic’s known patterns of spreading, keeping schools open should have been the foremost priority for the Indian government.

It has been noticed, over several years, that students who drop out of school in India rarely return at a later stage in life. Their chances at social mobility are affected significantly, and they are stuck doing low productivity jobs such as being a car mechanic or a repair technician: Their literacy helps them with these jobs which involve a lot of traveling in the hot sun but no gaining of knowledge or marketable skills. Their limited experience with formal education limits their ability to grow further, such as setting up their own repair shop or employing a few people in order to run a service business.

The advent of the gig economy has been seen as a boon by many, because most workers in the gig economy need only a driving license and do not need any skills as such. But this could end up becoming the reason for an increasingly literate society to regress back to its illiterate origins. The temptation of quick money made by delivering orders for food delivery apps often overcomes the long-term benefits of education. The Indian government should clamp down on the gig economy by introducing strict requirements for contractual workers, including limits on working time and minimum wages irrespective of demand.

Adaptation of Imported Technologies

This is one of the key concepts in this book. Due to reasons that Allen goes into in detail, the “Standard Model of Going from Middle Income to Higher Income” is no longer viable.

Here’s an obvious statement: Inventions can not be transported as-is across time or space. The new technology must be adapted to local conditions before it can be used widely and create economic development. How many governments are nimble enough to notice this? Not many, as Allen elaborates.

Japan was a country which had a lot of cheap labor. Labor saving inventions were discarded immediately, because they did not solve any existing problem. Instead, while the cotton mill was powered by steam in England because coal was cheap there, it was powered by humans in Japan. While mills were run for only 12 hours a day in England due to the culture of labor laws there, in Japan, the mills were run for 24 hours through 2 shifts of employees. (Once again, as labor was cheap in Japan, it made sense to hire people. Whereas in England, the limiting factor was labor and there was no point in running the mill for 24 hours because the extra labor was too expensive for the final product to be cost-effectively produced.)

India, on the other hand, was a part of the British Empire. India did not implement the 24-hour mill adaptation. Thus, even though labor was cheap in India, the mills were idle for half the day. This hampered the growth of the cotton industry in India, as the capital invested in these mills sat unused for half the time and took twice as long to pay back dividends to their owners.

In his discussion of why industrialization began in Europe and why England had an advantage, Allen points that England had 2 things which were not there in any other Northern European economy:

Cheap capital (to invest in new technologies and machines)
Cheap coal (to power the new technologies and machines)

These two advantages were decisive and enabled the start of the industrialization. On top of this, labor was expensive relative to capital and coal. So, businesses found it lucrative to invest capital in machines and train their workers and increase each worker’s productivity, as compared to hiring more workers.

Big Push Industrialization

As we saw, the Standard model for economic developments is defunct now. So, what is the solution?

Allen is optimistic: He says that Asia’s economies have grown through Big-Push Industrialization and that other economies can, too.

The structure of Big Push Industrialization is inherently unstable because it is based on a leap of faith.

Procure cheap capital
Build everything that is required for industrialization simultaneously (i.e. Capital is invested in steel production and automobile production at the same time.) Make a leap of faith that the dependencies of every effort will be completed in time.
Institute a planning authority that reinforces this faith. (i.e. It convinces car manufacturers that the steel they require will be produced domestically by the time they have figured out how to produce a car. It allocates funds to the steel industry to make sure that this promise is actually met.)

4 Asian economies employed this model successfully and closed the gap to the West in a single generation: Japan, China, Taiwan, and South Korea.

Japan

After World War 2, Japan closed the gap to the West in a single generation. Between 1950 and 1990, average annual growth rate was 5.9%.

The products that were being produced were steel and automobiles. State-of-the-art steel production technologies were imported. These technologies were capital intensive, but making a leap of faith, the capital was invested with the belief that there would be a market for the finished good. When the steel was used to produce automobiles, these factories were forced by central planning to be of at least Minimum Efficient Size. The produced cars were exported to America and caused a crash for domestic automobile production in the US.

This would have been a terrible result for any other economy, as the US could have easily imposed import tariffs and prevented Japanese cars from appearing on US roads. But Japan’s delicate position in Asia as a US outpost guaranteed its freedom to trade its goods in the US. After the Second World War, America had emerged as the most competitive economy. Japan’s fortune in securing its place as a seller in the American economy ensured demand for its cars.

China

No modern success story comes close to China’s. Production per capita in China went from $284 in 1988 to $10,500 in 2020. This 36-fold increase in 32 years is phenomenal and unmatched in any other economy of comparable size.

These are not just statistics, either. The effect on people has been stunning too. The percentage of population living below the global poverty line of $1.90 per day went down from 66.3% in 1990 to 0.1% in 2019 in China. Meanwhile, during the same period in India, the value went down from 50.6% to 22.5%.⁵

Switching from percentages to absolutes, we see the number of people who are affected. This table shows the number of people who were below the poverty line in 1993 and 2011.⁶

Country	1993	2011
China	668 million	106 million
India	441 million	281 million

In 1992, Central Planning was abolished in China. The remaining state forces guide the energy and heavy industry economy, which continue to be central to China’s growth. Both of these have grown constantly since the 1970s. Per capita income grew a whopping 6.7% every year between 1978 and 2006.⁷

Communal ownership of land meant that agricultural progress and its successes were more equitably shared across China. In India, small farms were pushed out by businessmen who had easier access to capital and could employ the latest innovations easily. China has managed to keep farms of all sizes alive, while small farms in India have retreated back to a subsistence level of life. The government has also had to step in and invest a large chunk of its capital into supporting farmers, the largest voting bloc in India.

As China comes closer and closer to the technological frontier, the state has been loosening its grip over enterprises so that market factors can gain control and enterprises are nimble to changes in the market. If this trend of adaptation continues, then China’s businesses and the Chinese Communist Party should be able to navigate the next phase of the country’s economic development successfully; possibly returning China to the place it occupied in the world, before Vasco De Gama and Columbus set out to find the new world, as the largest manufacturing economy in the world.

Africa’s Woes

The most popular argument I have read about why Africa remains the least industrialized continent relates to geography and its history. Allen puts forth a similar convincing argument: Geography and Colonialism.

Geography matters a lot. And in particular, the proximity to Northern Europe mattered the most. The ability to have a flourishing sea trade with Europe was the deciding factor for growth after the Industrial Revolution because Northern Europe was the most advanced region on the planet and reliable demand for your goods there would be a confirmed ticket into the global economy. However, the sea trade between Africa and Europe never flourished because the distances were prohibitive. The same fact affected South America. South America’s sea ports were farther from Northern Europe than those on North America’s Eastern coast (such as Philadelphia.) So, if the same product was to be exported from both ports, the demand would undoubtedly be higher for the export from Philadelphia because it was closer to Europe.

Colonialism in Africa started in 1800s and split up the continent by the late 1800s. The most common problem was that European imperialists split up African countries into cities and tribal regions. The cities were ruled directly by the imperialists, whereas the tribal regions had village chiefs as before. The imperialists also set up low tariffs to ensure that European goods would be imported and sold to Africans, and nothing would be produced locally. The effects of colonialism were much worse in Africa than in any other part of the world.

Land expropriation was also another huge problem for the African natives. Before the imperialists, land in Africa was basically worthless. Little effort was required to cultivate the land, so most communities were able to sustain subsistence levels of living without much effort. However, once the Europeans had established their rule on the continent, they started taking land away from the natives through imperialist decrees. For e.g., the British passed an act which gave 2/3rds of South Africa’s population the right to lease only 7% of the land in that country.

So, a combination of low tariffs and low wages have pushed Africa into a trap which it has found impossible to get out of. Wages remain low, productivity remains low too. Most of the cocoa in the world is produced in Africa, but export prices continue to be low. There is no incentive to improve productivity or to employ technology instead of manual labor when wages are low. Africa’s story is a depressing one: It is the story of a continent which was unable to escape the Middle Income Trap.

Values in 1990 US dollar. The poverty line is $365. ↩ ↩²
“40 pounds” in Allen’s original text. ↩
“1 pound” in Allen’s original text. ↩
The accurate statement here would be that per-capita value addition to GMV does not rise. The absolute amount of product produced does increase owing simply to medical expansion increasing the life expectancy of people and due to population growth. ↩
World Bank data. ↩
2011 was the year of the last census in India. No data for poverty in India after this census has been released. The choice of 1993 and 2011 is because these are the only 2 years when data is available on World Bank for both India and China. I made the calculations of “number of people in poverty” by multiplying data from the World Bank for “Poverty headcount ratio (% of population)” and “Total population”. (TSVs of Calculation: China and India: Poverty Headcount (TSV), China’s National Income Per Capita (TSV)) ↩
World Bank data. ↩

Using TikTok for 1 Month

2022-07-18T00:00:00+00:00

TikTok is the most talked about social media platform now. Articles about the impact it is having are published regularly in mainstream publications. People are constantly writing editorials about what it is, how it became popular, how it is owned by a Chinese company (and thus, controlled by the Chinese government), etc. I had read enough of it that I wanted to give it a try. I wanted to be a viewer on TikTok for a week (originally) to understand what it is really like. I ended up being on it for about a month. This was because I was not completely convinced that I had used all the features in the product after a week. Then, I gradually realized what was really going on: TikTok is incredibly simple; it requires no interaction from the user except for scrolling down. It is built with a singular focus on convincing people to imitate the trend that is “going viral” at any given point. There are many moving elements around the screen, and some of them are integrations with other businesses, which TikTok or the creator possibly makes money from. I think that TikTok is popular today because it solved a content creator’s biggest problem: The requirement to be original. Nothing is original on TikTok. Everything is a copy of a copy of a copy of a copy. But these are not copies. They are participants in a trend.

The First Day

After installing the app, signing up, and following some accounts, I saw that there were two tabs on the app. There was a Discover tab at the bottom of the screen. The search bar at the top had some searches which were marked as “trending.” Every time I opened the app, it opened on the “For You” page, which had a video from a user that I did not follow yet. The “Following” page which has videos from the people that I follow was deliberately harder to access than this “For You” page, where TikTok’s algorithm puts content it thinks you want to see (and possibly will keep you hooked.)

The simplicity of the app was stunning at first. I don’t say this lightly. I like plain text and spend most of my time inside a text editor or in Firefox’ “Reader mode.” While watching Video, I use the YouTube Unhook extension, which removes everything but the video’s title and the video from the screen. TikTok appeared simple to me despite my meticulous attempt to exclude moving elements from tools I use daily.

There was no text in TikTok. You can not read anything; there was nothing to read. You could see the first couple lines of the poster’s description, but only upto the 10th word or so. This was useless anyway because it generally contained the same hashtags: #foryou #fyp. This makes literacy and option for TikTok users. This is also unlike other platforms where long essays about the topic du jour are common. So, TikTok is really just a stream of videos which one would consume, much as they would watch TV. There were the quintessential “Like”, “Save” and “Share” button. Every video was set to some music, and the name of the music was in a chyron at the bottom of the screen.

You can search for people if you want, but it was clear to me that most people were meant to stay inside the For You page and discover videos there. For a few minutes, I was not sure what I was supposed to do. I have not watched TV in a long time, and I am used to watching content on demand. Being shown arbitrary videos and not doing anything to tailor the experience felt weird. This app was different: No input was required from me. This is perhaps the most jarring feature of this application for people who are used to a non-TV world where you don’t watch anything you don’t want to and content flows only after you enter a prompt of some sort. Even YouTube has text that you can read, search strings that you must enter, and video tiles that you must click before the content starts flowing. That TikTok requires its users to do nothing at all is a hearkening back to the early days of radio and TV: Turning on the device opens the firehose.

The Next Few Days

Over the next few days, I got used to the daily ritual. At some point during the day when I had a couple of minutes free (such as when I was waiting for some food to heat up), I would open the TikTok application. It would always open on the For You page, which was the first thing about the app I found to be annoying. I realized that the “Following” feature exists only to provide a “familiar” ground for users who are coming from other applications. TikTok content creators do not expect to have a loyal following. They expect their content to “go viral.”

I would swipe left to the Following tab and see content from the people I follow. One thing that is (probably) intentional is that consecutive videos are often set to the same music. They are from two different creators, using the same audio, and trying to do the same kind of dance in the video. So, each creator’s individuality rarely comes through: Indeed, that is not the point and the platform is built to emphasize that to the user.

There is no pretense of the videos being spontaneous. Every video is studied and well-rehearsed. The tempo of some of the songs is so high that it is nearly impossible to perform any dance satisfyingly on the first or even the tenth try. A lot of the dances that are performed are halfhearted attempts from a user who is in a public space and uncomfortably glancing around at involuntary spectators (a cafe or a park or a school). TikTok has not yet gotten rid of this last facet of inhibition.

Some users are using the platform to post vlogs (video blogs). These are a few minutes long and well-edited; the cuts are fast and a lot happens in a single video. There is little the user can notice well enough to comment on. (There is no “contemplative” sequence where the user might think about the content that has just been shown to them. There are no monologues where the creator really lets the viewer in. It is like being in the lobby of a busy office and watching a lot of deal closures and people handshaking, but not knowing exactly what people are happy / sad / excited about.)

Solving the “Requirement of Originality” Problem

The rise of social media was a boon for people who are hoping to become popular and make money off that popularity. It was a bane for people who are already popular or are going to be popular through traditional routes: reality TV shows, movies, music, models. The problem was that the mid-range celebrity had to now do this on top of everything else that they are already expected to do: Create content on social media, have a lively feed for brands to see, and a loyal following for brands to want to tap into. The number of followers is important for people who want to star in their big break movie or participate in a photo shoot for a large brand or appear on the cover of a magazine. The public’s adoration online is a proxy for competence among the agents who are looking for new talent. Would you be paid to promote the Fyre festival if you did not have a large following?

This group of “new wave” celebrities has to constantly make new content to stay relevant. Repetition or reposting will not cut it. They must create “original content” to retain their following. TikTok solved this problem and gave all mid-range celebrities a way out of the content creation rat race. What if copying was not taboo? What if copying an ongoing trend was expected by users and rewarded by brands? What if the stakes of not creating a new trend were lowered enough to make it not worth the time of mid-range celebrities to come up with new trends?

Who started the ALS ice bucket challenge? I don’t know. I know what the challenge is. I know what ALS is. The challenge ostensibly served its purpose. And it provided some ready made content creation opportunities for those who are forced, due to their profession, to create content.

TikTok provides these readymade content creation opportunities on an hourly basis. A mid-range celebrity can maintain their following by practicing the dance that is trending right now, and performing it for their followers. They don’t have to come up with the dance. While the creation of content might still take the same amount of time, the forced need to be creative is done away with; as is the risk of messing up and offending your followers, a route many a celebrity have taken with the wrong choice of words.

In a few days of intense use¹, I realized that this app gets rid of most of the “fluff” that is expected from content on other social media platforms:

Content should be original
Content should have inherent meaning or purpose. The purpose can not be solely “I want this to go viral.”
Content should be discoverable through the user’s preferred path. Companies will build elaborate search infrastructure. What if users understood, through product design, that searching was discouraged? Please do not resist.
Content should be spontaneous

So, what remains? The main purpose of all content on TikTok is to “go viral.” On TikTok, even virality’s definition is constantly changing. Trends used to be a few weeks long. Now, trends come and go in a couple days. So, if you were to not check your phone for a weekend, it is totally possible that you completely missed a trend. Also, you will never know that the trend ever existed because the next trend is probably already popular and users are posting content from that trend.

In a strange way, TikTok actually makes the “Fear of Missing Out” on social media less “world-ending,” even for heavy creators and users. When you don’t know what you missed, there is no reason to be really guilty about not using the app for a few days. However, TikTok’s algorithm is known to reward consistency. If you consistently provide free content to TikTok so that they can show it to people and make money off it, they will put you in their feeds. So, the FOMO for users is reduced significantly, while the FOMO for creators might be reduced only marginally.

Absurd and Unnecessary Sexualization of Content

A lot of content on TikTok is strangely sexual; that is, the content is much more sexual than the demands of the content are. A dance that can be performed anywhere is performed at the beach. A video that can be shot sitting still is shot while the subject is cutting a fruit in the kitchen. A video that can be shot at any time of the day is shot when the subject is out on a run or in a gym. A lot of content on TikTok is created by women. I am not sure whether it was my bias when selecting people to follow or whether the content creators on TikTok are mostly female; mirroring the general fact that men are heavier users of video platforms such as YouTube.

The hypersexualized nature of content on the platform has been noticed by others. An alarmist article from the conservative-leaning Wall Street Journal was the only systematic study of the type of content shown on the “For You” page that I could find. It mentions that TikTok has loosened its policy and started allowing more sexual content in its feeds:

Policing content has been complicated by the company’s decisions in recent years to loosen some restrictions in the U.S., including around skin exposure and bikinis, according to several former executives and content moderators. The result has been more sexualized videos on the platform, the people said.

One final comment that I would like to make is the apparent dissonance of feminist-leaning content on TikTok. This is undoubtedly a sign of the increased prevalence of feminist viewpoints in wealthy pockets of society. But how can one explain the self-objectification that is prevalent on TikTok?

I remain confused by this question. Amia Srinivasan, the feminist thinker, who recently published her collection of essays “The Right to Sex” tackles this in her essay, “Talking to My Students About Porn”, Srinivasan explains how girls and women in the 2000s are outspoken about being feminists and well-versed in identifying the injustices that are happening around (and to) them under the auspices of patriarchy. They are articulate in their expression of dissatisfaction with the status-quo. Despite this improvement in the feminist consciousness level among women, their sexual conditions are worse than before.

The young women Orenstein discusses in Girls & Sex would have known, unlike my younger self, exactly what to say. They would not have been ashamed, as I and all my friends were, to call themselves feminists. How should we understand the relation between this raised state of feminist consciousness among young women, and what appear to be their worsening sexual conditions: increased objectification, intensified body expectations, decreasing pleasure, and shrinking options for sex on their terms? Perhaps girls and young women are becoming more feminist because their worsening circumstances demand it. Or perhaps, as Orenstein suggests, feminist consciousness is for many young women a mode of false consciousness, which plays into the hands of the very system of sexual subordination they take themselves to be opposing. Does a discourse of sexual empowerment and autonomy mask something darker and unfree?

– Amia Srinivasan in her essay “Talking to My Students About Porn.”

Why Does TikTok Exist?

Money. Why do gossip magazines exist? Why do “storytime” channels exist on YouTube? Why do celebrities participate in 72 Questions?

A quick look at the way TikTok makes money shows us who the real users are: The brands who pay creators to promote their products. TikTok is a private company and not much is known about the business, except for what can be garnered from the public statements of creators on the app. The app remains free of ads as of July 2022.²

There is a lot of discourse on the Internet about how the real user is the advertiser and the users who are using the product are pawns whose data is sold to the advertisers by the platform. I won’t belabor the reader by repeating this discourse. This article from 2017 explains the basics well.

On TikTok, as on other platforms, content creators are paid for promoting products and brands. These brands including fast-fashion brands like SheIn (which deserves a separate post of its own) or other clothing brands like Sephora.

The Traditional Celebrities Takeover is Not Coming to TikTok

Every new social media platform that came around was quickly taken over by celebrities and news outlets. Facebook and Twitter feeds are almost always full of posts from these media outlets and personalities who employ large teams churning out content professionally and at a volume no individual user or group of individual users can hope to match. Why hasn’t this happened on TikTok yet?

I think the reason lies in the fact that there is no “reading” on TikTok. Unlike Facebook, Twitter or YouTube, TikTok creators are not incentivized to send people out of the app. So, while TikTok itself might show “promotions” which might send users out of the app (briefly), the same feature is not extended to users. The description is too short to convey anything of value and it is incredibly hard to create news or celebrity content that will sit well between two videos of a subject dancing in the foreground and a fast-paced Pop song in the background. The cost of bad content is steep because the next video is always just a flick away.

Goodbye Pi-hole. Hello CoreDNS.

2022-06-12T00:00:00+00:00

I understand the economics behind advertising. Advertising can be used to subsidize content. However, I refuse to accept that Bloomberg.com needs to show advertisements to a paying subscriber.¹ This comes down to personal preference: If you believe that advertisements are a net good and support the work that you are viewing “for free”, so be it. I do not believe that. If I must pay for content that is ostensibly “free” by having my attention distracted by an advertisement which I did not ask for and am not interested in, then I reserve the right to use every tool in my arsenal to avoid seeing the advertisement. All of this is just a preamble to a recent change I made in one of the central components that I use to browse the Internet: my DNS server. I switched from Pi-hole, a popular adblocking DNS server, to CoreDNS and a plugin for ad blocking which I wrote. This is a post about why I decided to reinvent the wheel.

Why Should You Run Your Own DNS Server?

DNS (Domain Name System) is a key part of the infrastructure that makes up the backbone of the Internet. It is also the most decentralized part of it. Most people use a DNS server which is a few kilometers away from them. Internet Service Providers (ISP) will generally send default configuration to their clients which will point them to DNS servers run by the ISP. This allows the ISP to make a list of websites that their users are using and possibly use it as leverage when negotiating with service providers about preferential treatment. However, the clients that users use to browse the Internet (their web browser or their computer’s operating system) have the ability to override this setting and specify their own DNS server. Public DNS servers have been on the rise: Google’s 8.8.8.8 and Cloudflare’s 1.1.1.1 are popular alternatives. These DNS servers have a significant improvement: They are far more reliable compared to your local ISP’s DNS server. However, data about your Internet traffic continues to be sent to someone. To make the situation worse, DNS traffic is mostly served over UDP. UDP is an insecure protocol which does not protect information in transit. (Not “HTTPS,” if you will.) So, your ISP still has access to the list of websites you are visiting. The solution to all of these problems is hosting your own DNS server on a Raspberry Pi computer inside your home. (A recent improvement over the old design of UDP-based DNS is DNS-over-HTTPS. This ensures that DNS traffic can not be snooped upon, in transit.)

So, there’s one advantage: Your DNS request, which contains the name of the website that you are visiting, does not leave your home. Your ISP has no clue where your traffic is going. (They can figure it out using the IP address that your packet is headed for, but this is harder and ISPs have a tendency to aim for the low-hanging fruit and not spend too much time trying to track their users, if the information is not handed to them on a platter.)

There is another advantage: If you don’t like the domain inside the DNS request, then you can stop the request in its tracks and prevent the client from fetching what it wants to fetch. Your browser loads a page (for example, a news website). The news website has a script which will call an advertising server and request the ad which will be placed in the box in between paragraphs. If you could identify just these requests to the advertising server and block them somehow, then the website will not be able to show you ads!

One idea is to create a list of the IP addresses that are used by advertising servers around the world and block any request which is bound to these servers. This is hard and not reliable enough because IP addresses change a lot and the process for changing them can be easily automated.

Another idea is to list the domains which these servers are on and block any DNS request for these domains! This approach is convenient and reliable: Changing domains is hard and even if domains are changed, if you don’t want anything on the domain, then you can simply add the new domain to your blocklist. (Changing domains is hard primarily because each new domain has to be registered with a domain registrar and this requires a small fee. The fee must be paid and payment to vendors on demand is hard to automate. Also, so few users are using ad-blocking DNS servers that the return-on-investment for an automated domain changing system is limited.)

So, we have a good approach. But we are missing one key component: The list of domains which we want to block. There are many excellent lists out there and I recommend this one. It lists about 1 million domains and it is updated frequently. It is an aggregation of other lists and works well. (I have used it along with another list for the past several months, and I have not had any domain which I wanted to send traffic to blocked due to a domain being mistakenly added to the list.²)

Now, we have a list of domains that we want to block and the approach that we are going to use to block the domains (a DNS server which can look at this list and block requests which match a domain in the blocklist.) Now, we need to search for the software which will implement our solution.

Why CoreDNS?

There are many options. The most popular one is Pi-hole, I used it for a few months and realized that I could not continue using it because I disagreed with its GUI-centric design philosophy and the effect that this philosophy had on its purpose: being a good DNS server. CoreDNS is a DNS server which puts plugins front-and-center and encourages its users to write their own plugins. It is written in a programming language that I like and am familiar with, and I don’t mind maintaining a plugin in the long term.

Pi-hole

Pi-hole is written in the programming language C. I chose it, as I believed that it was built for running on a Raspberry Pi. (I did not research how it was built or why it would run well on a Raspberry Pi.) It was fairly easy to set up on a Raspberry Pi. There were some manual steps during the setup process and it could not be fully automated. However, this was not a big problem for me because I did not believe that I would need to repeat this setup process.

I started noticing problems with Pi-hole almost immediately after setting it up. The main problem seemed to be with its statistics reporting module. The sane way to report statistics is to provide an API endpoint which can be scraped periodically. This API endpoint should either take the starting time for statistics or have a configurable time period, which can be passed as a parameter to the request. Pi-hole had a strange 24 hour pre-configured time window. I was not sure why it was 24 hours or why it is pre-configured.

I was convinced that this was going to be a deal-breaker for me. The only thing I wanted my DNS server to report was the number of requests that it blocked and the number of requests that it forwarded to the upstream DNS server. This value could be wrangled out of Pi-hole and I figured out how to do it. I had to dive deep into Pi-hole’s C code. There isn’t that much of it and I was able to read it all over a few hours. I had to make 2 pull requests to use the existing configuration variables. These configuration variables were not respected by the application code; a bad sign for a well-maintained codebase.

With these patches, I had managed to get the statistics that I wanted. Perhaps, I should have stopped here.

However, the more I thought about what Pi-hole was really doing, I realized that its only reason for existence was statistics collection. Indeed, the whole repository was a large wrapper around Unbound, a full-featured DNS resolver. All Pi-hole did was maintain a few counters which it incremented based on the requests that it received/blocked. The statistics that it collected were supposed to be displayed on a GUI dashboard. This dashboard is the reason for the mysterious “24 hour” value above. The code for these counters was not stellar either: There were many global variables and there seemed to be very little of the neat dependency injection that I am used to.

I did not like this design. A blocklist-enabled DNS server should do one thing and one thing well³: Serve DNS requests and block the ones that the user does not want to serve. To plaster a GUI onto this server and change the code inside to fit it to the needs of the GUI is a bad idea, in my opinion. If users want a GUI, then they should use the statistics that are reported by the DNS server and make their own GUI. At the very least, the GUI should be an optional part of the DNS server and no code or decisions made inside the DNS server should be to please the GUI user.

Pi-hole did not follow any of these principles. Most of the code inside the repository was primarily to make the GUI better looking. One of the patches I made ended up creating a bug for most users of the Pi-hole GUI because it changed the way that time slots were stored. Instead of storing statistics for the “24 hours until now” when the server starts, I proposed storing the “24 hours from the start of the server.” My approach would make sense if the server had to store the statistics only to report it to the user as a number. If the statistics are part of a GUI, then when the server starts, the user will see a jarring empty graph with the current time on the left side of the horizontal axis, with the future 24 hours with no data.

This bug which my innocent PR to improve statistics reporting had caused put me over the top. I decided to change my DNS software because I could not live with this C code which was not designed in a way that I approved of and had a lot of code to satisfy GUI users, when I knew for a fact that I was never going to use the GUI bundled with a DNS server.

CoreDNS

CoreDNS is a DNS server written in Golang. It is not a recursive DNS resolver. Indeed, it does not do anything related to the recursive process required to resole DNS queries. However, it does everything that is essential before and after this core part of the DNS resolution process. Most importantly, CoreDNS is built with plugins in mind; it provides first-class support for preexisting plugins as well as plugins written by the user.

Another advantage to keep in mind is that CoreDNS has excellent built-in support for metrics collection. It is designed to be run inside environments like Kubernetes, where metrics collection is not an afterthought. Specifically, CoreDNS integrates with the popular monitoring server Prometheus. More on this later.

The plugin API is intuitive. Every DNS request passes through a series of plugins. By controlling the order in which these plugins operate, it is possible to change the input or the output of these DNS requests. And any plugin can return a response to the user. All the heavy-lifting required to keep track of plugins, integrate with each plugin and call them at the appropriate time, identify whether the plugin is going to respond or simply fall through to the next plugin is done by CoreDNS. Plugin writers have to care about a single function: This function receives a DNS request as input, and the plugin writer’s responsibility is to either set a response to this request or to return an error code indicating that the request should not be passed to the next plugin in the plugin chain.

Another (incidental) advantage is that I have been using Golang at work for the past 4 years. My familiarity with the programming language is quite high and I was happy to write and maintain a CoreDNS plugin. I hoped to learn more about the nitty-grity of DNS implementations, and common pitfalls that one might fall into along the way.

So, over 2 days, I wrote a CoreDNS plugin to block domains based on a file and Ansible configuration to deploy this plugin alongwith the main CoreDNS application to my Raspberry Pi. The process of writing the plugin was good. I started with the coredns/example plugin provided by CoreDNS’ authors. This plugin shows other plugin writers what to think about and how to architect their plugin. After a few hours of understanding how plugins really worked, I was able to get a working prototype with a hardcoded list of domains. Switching out the hardcoded source of domains with a file on the local file store was easy. The most confusing part of the process was understanding where the order of plugin execution was defined. I understood this process after reading the excellent blog post from CoreDNS’ authors: How Queries are Processed in CoreDNS.

Other Options

AdGuardDNS is the closest alternative to what I built. The DNS server is a fork of CoreDNS and it uses a bunch of plugins built and maintained by AdGuard. I don’t have a good reason for not using this. One of my friends uses this on their Raspberry Pi and they recommended it to me. I looked into it and decided not to use it mostly on a whim: AdGuard is a business that sells an AdBlocker and VPN software.⁴ The DNS server is a part of their offerings. While the DNS server is open source right now, there is some discussion about how future features might not be added to their open source offering, prioritizing the paid alternative instead.

We use AdGuard DNS functionality as a part of other AdGuard software, most of which are distributed on a pay-to-use basis. We might also develop a paid version of AdGuard DNS based on the current one, more advanced and with more features. – AdguardTeam/AdguardDNS’ README. Section: “Why is AdGuard DNS free? What’s the catch?”

My coworker @execjosh’s mydns is another blocklist-enabled DNS server. It has a lot of logic which was not a part of my CoreDNS plugin because all the request handling was outsourced to DNS. I used mydns as a reference for implementing the tricky parts of DNS: Setting the reply and understanding more about DNS question types/classes.

Improved Metrics Collection

So, did my monitoring problem get solved? Yes. Bonus: I did not have to write any code to solve it.

CoreDNS comes with a metrics plugin. This plugin reports metrics in the Prometheus format. Having worked with Prometheus servers inside Kubernetes for the past few years, I am familiar with this format and I marvel at its simplicity.

Getting the number of blocked and forwarded requests from a Prometheus-compliant metrics endpoint is very very simple.

pi@dns-1:~ $ curl localhost:9153/metrics -s | grep dns_responses_total
# HELP coredns_dns_responses_total Counter of response status codes.
# TYPE coredns_dns_responses_total counter
coredns_dns_responses_total{plugin="",rcode="SERVFAIL",server="dns://:53",zone="."} 458
coredns_dns_responses_total{plugin="blocker",rcode="NOERROR",server="dns://:53",zone="."} 5191
coredns_dns_responses_total{plugin="forward",rcode="NOERROR",server="dns://:53",zone="."} 218955
coredns_dns_responses_total{plugin="forward",rcode="NXDOMAIN",server="dns://:53",zone="."} 2073
coredns_dns_responses_total{plugin="forward",rcode="REFUSED",server="dns://:53",zone="."} 2232
coredns_dns_responses_total{plugin="forward",rcode="SERVFAIL",server="dns://:53",zone="."} 426

Using the plugin= part of this output, I can see that 5191 DNS requests were blocked by the server. 218,955 DNS requests were forwarded to the upstream DNS server (an Unbound instance running on the same Raspberry Pi.) Forwarded requests faced some errors like “Non-Existent Domain” (NXDOMAIN) or “Server Failure” (SERVFAIL).

If I were to run Prometheus on the Raspberry Pi, I would be able to collect these metrics easily and use Grafana to visualize them. However, I don’t think that I will be doing that. The Raspberry Pi uses flash memory and is severely limited in terms of disk read/write throughput. The DNS server itself will rarely read from disk; it needs to read the blocklist file from disk periodically. I am interested in only one metric from the CoreDNS server, so I believe that I will continue to use an external monitoring tool such as Mackerel or a local Round Robin Database, which is more typical for networking infrastructure.

Bloomberg subscriptions cost $35 per month. ↩
The only domain that came close was related to Bloomberg and its emails. This domain was included in the lists as a domain that serves ads, but it was a domain which is used for links inside newsletters sent by Bloomberg via email, and is a tracking domain indeed. It redirects to the main Bloomberg.com website. ↩
Unix philosophy ↩
The essay Don’t Use VPN Services accurately captures my feelings about what VPN services really offer and whether people should use services that they don’t control. ↩

Notes and Review - A Burning (Majumdar)

2022-05-29T00:00:00+00:00

A Burning is ostensibly about 3 people. Their lives are entwined in rather banal ways: teacher, a former student, a young woman helping a slum dweller learn English, a transgender beggar who is learning acting, an acting teacher who has never had a real audience. Their hopes and dreams are conventional: The young woman gets into a good school and manages to escape her past and moves solidly into the middle class. The slum dweller hopes to learn acting and make it big in the movie industry. Under this shroud of normalcy, and sometimes using it as a tool, Majumdar captures the depressing nature of the life they lead. And these characters stand-in for a whole group of people; the group that has missed the social mobility bus; the bus was packed in the 1970s with first-time engineers and doctors and lies overturned on the side of the highway today.

I lived for 5 years in West Bengal during college. The college was situated inside an insulated cocoon where the outside world rarely interfered. Every time I went to Kolkata, I had to go through Howrah Station. This railway station has been around since colonial times and it has remained mostly unchanged during that time. The abject poverty that is prevalent in the state and the stark income inequalities which are prevalent across India crystallized into a solid object which I had to wrestle with, every time I went through the railway station on my way to a waiting Uber driver. Majumdar has captured all of this terribly well.

For all the astute observations about poverty as a way of life, and the irrational character of social mobility, this novel is surprisingly dispassionate. The chapters titled “Interlude” are testament to this feature: The stories inside these chapters are the most touching renderings of poverty I have ever read. In particular, the story in which 2 young men are demanded to pay an exorbitant entry fee to enter a mall, a fee that seemingly does not apply to plump women who look rich, has remained in my mind ever since I read it.

So Raju and I stepped away from the entrance. We looked at each other. Neither of us wanted to say it. So Raju clapped my back and I smacked his shoulder, and we went to the syrup-ice stall and had some orange syrup-ice. Then we went back to work. Him to his house painting, that paint-smelling turban on his head again. Me to my electrician’s shop. It was giving me pain in my wrist, pain in my thumb. At least the syrup-ice was delicious.

This is the shadow of discrimination that inequality inevitably creates. While many of my middle-class and upper-middle-class acquaintances strongly believe that India has come a long way from the days of unemployment and poverty in the late 20th century, the improvements have been captured by a few segments. There has been limited society-wide improvement in the decades since.

All three characters have an arc of social mobility. This arc is defined by the sacrifice that they have to make. In the case of the soon-to-be politician, it is the witnessing of a mob lynching, fueled only by communal hatred. In the case of the soon-to-be actor, it is the abandonment of a benefactor and friend. The main character sacrifices her life to “answer” the demands of the public, that blame be placed upon someone.

[In the courtroom, the judge announced his ruling,] “And, on the other hand, we have the word of a hijra¹, an individual who begs on the streets for money, saying the defendant taught her English. Be that as it may”–the judge takes a deep breath, … –“it is clear that the defendant has long been disloyal to the values of this nation. The defendant has spoken clearly against the government, against the police, on the Internet, on Facebook dot-com. This lack of loyalty is not something to be taken lightly. It is its own strong piece of evidence. There is a case to be made, as well, for soothing the conscience of the city, of the country. The people demand justice.” He goes on.

These sacrifices are irrational. Towards the end of the book, that last sacrifice made by the protagonist enraged me. How can one person be sacrificed at the altar to pacify the demands of a nation for justice? Yet, what right do I have to be enraged? The first decade of the 21st century in India was ridden with terrorist attacks across the country. After each attack, security was increased and people (sometimes, communities) were sacrificed. The nature of these sacrifices is the same and if I am enraged by the fictional one, then I must battle with the injustice inherent in the non-fictional ones too.

The final notable feature of the novel is that the main incident in the novel, the burning of the train, takes up so little of the story. It is described. And the main character’s feelings about it and where she was in relation to the attack are also described. But we never get to see the damage that it has done. Our perspective is forcefully limited to the daily lives of the 3 main characters. Unfortunately, it includes such useless sources such as the nightly shows that PT Sir watches on his TV or what Lovely’s friends tell her about their opinions. In this, I think that the author has pointed towards the horrifying nature of life in a world of information abundance: What has happened might be important. What others are saying about it is always more important.

As tragedies engulf countries around the world in the form of inflation, food insecurity, heat waves, and war, there is a sense that this is business-as-usual. Today’s tragedy is the latest of a series of vignettes; certain to fade away when the next one appears.

P.S. One nice part of the novel that I appreciated was that one of the characters who is in the process of learning English, speaks in broken English throughout: (even in her inner monologues)

Even a dog which is looking like a wolf is enjoying the ride in AC comfort. All of them are ignoring me. The public is wanting blood. The media is wanting death. All around me, that is what people are saying.

This character, Lovely, is learning English and she makes the kind of mistakes that are characteristic of a beginner in India, prominently, the usage of the continuous tense everywhere.

The Hindi word for transgender. ↩

Gaslit on Twitter

2022-05-16T00:00:00+00:00

Gaslighting is a catch-all term. I understand it as the act of relentlessly bombarding a consumer of information with a large amount of irrelevant statistics and facts to such an extent that the victim can no longer convincingly explain (even to themselves) their fundamental beliefs. There is a lot of talk of “gaslighting” in media environments where there is a single source of news. In India, nightly news shows are a great example. It is also relevant in environments where a news source is seen as biased. The biased news source’ reporting is criticized using a bunch of unrelated facts.

On the 28th and 29th of March 2022, there was a 2-day strike call in India. This news was reported by the New York Times, and crucially, it was included in the “Daily Briefing” newsletter’s Asia edition on 28th March, 2022¹. I had not heard about the strike. (Not surprising, as I don’t live in India and have been more out of the loop than usual lately.) None of the people I know in India mentioned it to me over the previous weekend. When I read about the strike, I did not think that NYT was reporting a lie. I accepted it as fact and went to a few Indian news websites to read more about the strike. Quickly, I started questioning what was going on and why real people (not “trolls”) were saying absurd things online. I am not the first person to go through this disorienting experience and I won’t be the last. I want to talk about why I felt disoriented, how I battled with this feeling, and how I managed to resolve my disorientation by doing some research.

Part 1: Is It True?

There is nothing quite so humiliating and disempowering as trying to prove the truth. – Masha Gessen on “Why obvious lies make great propaganda”

I started scrounging around the websites of some well-known Indian news websites. I checked the website homepages for NDTV and Times of India. These two did not have any mention of the strike on the homepage. A 2-day strike call across all of India is a big deal. It can potentially shut down the economy completely and significantly affect people’s lives.

(For e.g. Banking and public transport unions had called for a strike. This could have affected anyone who was planning to take a bus to the train station on one of those two days or was planning to withdraw cash from a bank for a crucial purpose such as the purchase of a home or jewelry.²)

This is very newsworthy. Even if these news organizations had information which suggested that the strike’s effects would be limited, this is front page news. Not being able to find the word “strike” on the NDTV homepage is confusing.

After a few minutes of searching around for the terms using a search engine, and specifically the query site:ndtv.com india strike, I saw that the strike was very much real and happening across India. It was being covered by both the news organization whose home page I had checked; However, the links to these specific pages were not included on the homepage. So, the only way to reach these pages was from an external search engine.

I had confirmed that the strike was indeed happening. To find out more about its immediate effects, I went to Twitter. I was hoping to find some people who were as stunned as me by the (ostensibly deliberate) under-reporting of this story. At the very least, I wanted to find some users complaining about or reporting some of the disruptions caused by the strike. I did not find that.

Part 2: Here’s The Thing About Twitter

Twitter is not very relevant when it comes to learning about what Indians are thinking. Looking at just the raw numbers, it becomes apparent that a very small percentage of the total population is on Twitter.

Statistic	Count	Percentage of Population
India’s population³	1.40 billion	100%
Number of people on the Internet in India⁴	624 million	44.5%
Number of people on Twitter⁵	34 million	2.43%

2.43% need not necessarily be unrepresentative. But clearly, the exclusion of nearly half the population that does not have access to the internet makes the sample of users on Twitter an unrepresentative sample of the population.⁶

There are some other biases when it comes to the segment of the population that is on Twitter. Namely, they are almost certainly literate and at least partially proficient in English. (This last assumption might not hold true for much longer, as Twitter seems to be branching out into regional languages.) These add up to a good assumption that most people on Twitter are not using public transport daily (either because they can afford taxis, have private transport, or do not need to go anywhere on a daily basis because they are working-from-home) and that they probably never go to brick-and-mortar bank branches, doing most of their banking online. So, their opinions about a strike which would affect these two segments of daily life are of little value.

Nevertheless, I went to Twitter because this is also the only platform where open discussion is possible, except for group chats that one is a part of. These group chats are generally echo chambers and seldom contain opinions that I forcefully disagree with.

On Twitter, I did not find reaffirmations of my belief that this story was being under-reported. Quite the contrary.

It became clear soon enough that a New York Times tweet, one that mentioned the article about the strike in India, had become a flashpoint for all strike-related conversation on Twitter.

A two-day nationwide strike in India, involving both public and private sector workers, has disrupted transport and other services across the country.

Workers are protesting the government’s economic policies, including a privatization plan.

General Strike Throws India Into Confusion

– Twitter (March 28, 2022)

Around this tweet, there was much derision targeted at the NYT.⁷

Really ! When did this happen in my country ? Guys stop googling to write news , be on ground to write news.

– Twitter

There were some attempts at humor. But the meat of the derision clarified the biases of the population that is on Twitter.

Cars on the roads? There can not possibly be a strike of any kind.

Take a look at Kolkata today buddy. Wipe your eyes once again.

– Twitter

Traffic in Kalyan

– Twitter

Offices and shops open? There can not possibly be a strike of any kind.

Business as usual, shops are open, so are offices. Roads are jam-packed with vehicles.

Where is this so called bandh?

– Twitter

Why was I wading through all this? To find some mention of whether public transport and banking services were open, as the strike was targeting only those. One user reported that buses in Mumbai were on the road and plying as usual. This is the kind of useful discussion starter that I was expecting on the platform.

Where is the strike ? My kids went to school and are back . Even my maid came to work by BEST

– Twitter

Most of the tweets were just derisive towards the NYT; some mentioned articles about media biases against Asian and other lower-income countries; or about how the NYT did not have a reporting team in India and did not know what was going on on-the-ground. No one was engaged in any kind of real fact-finding about the strike and whether it was affecting someone else. Indeed, the self-centered-ness of some of these users is quite shocking.

Part 3: What Happened?

Reporting from the strike clarifies its impact. Here’s a catalog of the impacts with the relevant sources.

Banking services were affected nationally.

“As per our reports, in the Southern Grid functioning in Chennai, during our strike yesterday and today, about 6 lakh cheques/instruments worth about INR 5,000 crore could not be sent for clearance as branches did not function due to the strike,” All India Bank Employees Association (AIBEA) general secretary CH Venkatachalam said.

At the national level, about 20 lakh cheques worth about INR 18,000 crore could not be cleared, he claimed⁸.

The informal sector and small-scale industries were not impacted.

However, the Confederation of All India Traders (CAIT) said the strike called by a joint forum of trade unions has been a “total flop” in the informal sector.

In a statement, it also claimed there was no impact visible even in various industrial areas where small industries are having manufacturing activities⁹.

Kerala, a Southern state in India, seems to have been worst affected. This is not surprising; decades of CPI(M) rule in Kerala have created strong trade unions in the state. The same is true in West Bengal, where the Left Front was in power for 35 consecutive years between 1977 and 2011. (There is a common joke about West Bengal and strikes.)

State-run KSRTC buses did not operate while taxis, auto-rickshaws and private buses kept away from the roads across the state. Commercial vehicles including trucks and lorries are also not running expressing solidarity with the strike.

However, the essential services including milk, newspapers, hospitals, ambulances were not affected.¹⁰

Participation seems to have been significantly lower than the expected 200 million formal and informal workers that the unions were expecting.

Amarjeet Kaur, All Indian Trade Union Congress General Secretary, told PTI that they are expecting participation of over 20 crore formal and informal workers with mass mobilisation of workers across the country.¹¹

Here’s what happened: A bunch of trade unions issued a strike notice to nearly 200 million workers across India. This notice was reported upon by various news organizations within and outside India. This reporting did not have any unusual bias in favor or against India. The strike notice was not as effective as the unions initially expected. The strike’s impact was felt by a segment of the population, especially those in the states where unions are strong and the history of strikes is long. However, the people online who were sharing their opinion and experiences on Twitter were mostly unaffected by the strike. This lead to a belief that there was no strike and that reporting was reflecting the foreign media’s anti-India bias. These people did not take any effort to confirm whether the anecdotal evidence they had at hand was representative of other population segments. Once people started railing on NYT, Twitter became an echo chamber where no other opinion could break through. Once this pattern took hold, the story was no longer “Strike notice not as effective as unions initially predicted.” It became: “NYT blows strike in India out of proportion.”

I could not find the number of users who read this briefing. ↩
India’s ATM withdrawal rules are strict and do not allow withdrawals of more than Rs. 25,000 per day. ↩
India Population (Live) (Extrapolated from UN data) (Worldometers) ↩
Internet usage in India (Statista) ↩
Number of Twitter users in India from 2013 to 2019 (Statista) ↩
By comparison, in the US, 68 million people are on Twitter (20.5% of the population) and in Japan, 26 million people are estimated to be on Twitter (20.8% of the population). ↩
Going forward, I mention the text of a few tweets. However, I have decided not to publish the links to these tweets in this blog post. ↩
Banking Services Impacted On Second Day Of Nationwide Strike ↩
Nationwide Strike Hits Banking Operations, Transport Services ↩
Two-Day Strike Affects Normal Life in Kerala ↩
Trade Unions Call For Bharat Bandh Today, Tomorrow: Your 10-Point Guide ↩

"Absurd Customer Service" - Part 2, Suggestions

2022-05-01T00:00:00+00:00

I am not one to balk when it comes to writing rants and leaving it at that. However, when publishing rants, I think it is worthwhile to also think about the problems and try to figure out whether there is some solution to the problems that one is ranting about. Often, there is no such solution. If an easy solution did exist, there would be no need to find catharsis through the process of writing a rant. In the case of the Absurd Customer Service problem though, there are some solutions. Some of these are practical and can be adopted in our daily lives. While others are absurd; for the absurdity of the solution must mirror the absurdity of the problem.

(This is the second post in a 2-part series about absurd customer service experiences. You can read the first part here.)

Customer Service Agents Are People. Empower Them To Make Decisions.

As an example of good service, I will cite Amazon. Okay, I know. Amazon is an example of how capitalism has gone terribly wrong because their treatment of warehouse workers is questionable and they are openly opposed to unions. But I will cite Amazon’s Customer Service anyway because there is an important aspect of customer service which has been bulldozed by technology.

When you contact Amazon’s customer care and explain your situation to the person on the other end of the phone line or the chat message, they are listening. This basic act of empathy and attention is rare, with customer service being staffed by poorly paid, unmotivated agents who are not being paid enough to care and are being forced to work inhumane hours. After listening to the situation that you have gotten yourself in, the customer service agent is able to make decisions. They can decide to issue a refund for your order. They can contact the seller and ask them to replace what you ordered. Their resolution depends on your situation, but all of this stems from the fact that they trust your version of events and the customer service experience is biased to be good for the customer and biased against the seller. (Sellers will definitely complain about this bias. If I were to become an Amazon seller, I would probably cite Amazon as an example of bad customer service.)

This organization-level trust in the customer service agent is crucial. If you don’t trust and empower the people in the trenches to make decisions, you will frustrate the customer. As an organization, you have the power to strong-arm shady suppliers. The customer does not have this power. Customer retention can not be the reason you choose to have great customer service. Amazon does not provide good customer service because they are afraid of their customers going away. (You are going to a go to a store to buy a USB charging cable? What, are you crazy or something?) This move to provide good customer service must be driven by a profound desire to offer good service.

Customer Service Via Email

Most companies have moved away from the helpline number which can be called to talk to someone. This is okay. This is not the problem. If you have a customer support email with a median turn-around-time of 2 hours, it does not even matter whether you have a helpline number or not. Everyone wants to explain their situation to someone and be heard. If you give them a single email address and respond reliably to your customer’s queries, they will still remain overwhelmingly happy with your service. Do not rule out the phone: What can be explained in a 10 minute phone call can take up 100s of lines of text. Nevertheless, moving away from the phone is okay. Moving away from providing useful service is not.

Many companies have also taken to using chat bots. I think this is a bad move. Chat bots and chat with customer service agents gives the illusion of real time communication. But most people reaching out to customer service are probably not in standard situations which can be listed up and supplied as input to a chat bot software, or can be resolved quickly by an agent. It will probably take time to understand the situation, make a decision, get approvals for the decision, and communicate it to the user. All of this can not be completed in a 5-minute chat session in which you can not move away from your screen. And hopefully, you won’t close the tab that has the open chat with customer support.

Stay Away From The Absurd. Stay Away From The Shiny New Thing.

Historically, old-school capitalists were known for their conservative beliefs and their deep suspicion of the “new thing on the block.” Now, the equation is reversed. Capitalists want to employ the shiny new thing, as soon as it comes out. Everyone abandoned e-mail for Slack and helplines for SalesForce. Where are we now? 100s of unread threads on Slack every Monday morning and customer service inquiries which never get answered despite having a “reference number” automatically generated by your ticketing software.

There is a semi-popular adage about technology: When any new technology comes out, don’t be an early adopter of it and don’t be the last person to adopt it either. You have to choose your time and start using the technology somewhere in the middle; at a point when it has gotten sufficient traction to be well-understood and sufficient time to mature into a cost-effective product.

Until a few months ago, I believed that this kind of cost-benefit analysis was built into our economy. After all, decision makers demand huge amounts of data, analysis, and well-built presentations before deciding to delve into something. Why, then, do we end up with bad systems? I have come to the conclusion that when it comes to the shiny new thing, the new thing is so shiny that many decision makers will look past the blemishes just under the shiny surface and claim to see the vision at the core of the technology.

This has reached cult-like levels with crypto-currency and decentralization:

Cryptocurrency and Web 3 are the shiny new things. Yes, they have some problems right now. But hey! It’s early days. We are heading towards a glorious decentralized future.

Re: Your Mobile Number. Don’t Ask, Don’t Tell

Well, this is the hard one. For customers, don’t share your mobile number with the shops that you shop at. Why would you share your mobile number with a random cashier when you would think long and hard before sharing it with a potential mate? What is the value addition for you? Why do you want to give this information to a shop? How do you know the shop is not selling the database that it has built up of names and phone numbers to some sales agency? Our mobile numbers are the most direct ways to reach us; there is no filter. Do Not Disturb services offered by operators are famously bad. I think that we would love being in a world in which every email and every phone call we get is actually something that we are interested in. With email, we have methods to filter out the ones that we don’t want. Unfortunately, these mechanisms do not exist for mobile phones right now.

The nature of mobile telephony also makes it very hard to switch your phone easily. You would almost certainly have to spend several hours changing your phone number at the places that matter to you (banks, security services, online 2-Factor Authentication). We want to avoid doing this as much as possible. And the simplest way to do this is to simply not share your number.

Another possible method that I have heard of is to have some sort of a throwaway number which you can use everywhere that you don’t want to give your main phone number. But I think this is more cumbersome and fiscally wasteful. Now, you have two phone numbers to keep track of and there is no way to know when you will need which number. So, you have to carry both numbers along all the time. That is suboptimal.

I understand that organizations and businesses want to track their customers. They want to find out who is a loyal customer and give them deals or maybe they want to entice them to come back by sending them a message perilously close to their birthday or Valentine’s Day. In these cases, they do need some method of tracking customers and their purchases. But I think that this can be based on the points-based reward system. A basic point system would award you points based on your purchase, which can be kept track of preferably using some sort of paper card. You get extra points for your birthday, etc.

This system works very well. I can say this from my experience in Japan. Perhaps due to restrictions on the storage and sharing of personal information, most shops in Japan will ask you if you want a point card whenever you purchase something there. If you say no, they leave you be. (They will ask you the next time you shop there.) If you say yes, they will give you a small piece of paper or a coupon which can be used on your next purchase there. This is surprisingly effective. I have point cards from some of the chains that I use regularly for furniture, electronics, coffee beans. And I specifically look for these shops when I have to make some kind of purchase. This is the perfect situation for a customer; they are able to enjoy the benefits of being loyal to a chain of stores without divulging any personal information to the chain.

Technology Is Not A Silver Bullet

I think everyone needs to repeat this to themselves at least once a day. That is not hyperbole; it is a real suggestion for people who are making decisions about the adoption of new tools. Most cities in India have watchmen standing at the gates of apartment buildings. This has been a fixture of apartment buildings of all sizes since the early 2000s. In this landscape, there is hardly any security threat which the watchmen can’t identify or act on. Yet, a business such as this one is able to thrive. Why is that? Why should someone who has come to your apartment building to deliver a package to your house have to be approved on a mobile application before they can deliver the package that is addressed to you? This redundant verification is futile. It assumes that the watchman who is standing at the gate will not be able to make a judgment about whether the person and the package are legitimate. It is interesting to note that most buildings which introduce this application do not actually cut costs by getting rid of the watchmen who are guarding the building at the gate. This kind of application is introduced simply because technology is the solution. No one has noticed that there is a problem yet.

There is no simple way to convince people that the introduction of new technology is always a bad idea when it does not solve an existing problem. There are some people who want to venture out into the unpaved land and explore new ideas. I certainly do not want to try new technology simply because it exists. It is a waste of my time. And it is a waste of yours too. So, before you adopt the latest technology in customer service, ask yourself if you are solving a problem with it or whether you are indulging your desire to be known as a cutting edge organization. *No matter what technologies you introduce, the core requirement for good customer service will always be empathy; not a computer screen or a case reference.*

"Absurd Customer Service" - Part 1, The Problem

2022-04-30T00:00:00+00:00

Over a series of several occasions last year, I experienced the absurdity rife in modern customer service. I don’t think that things have always been like this. Customer service was a pain and you would often by stuck on 30 minute phone calls trying to explain your situation to the other person. But there was a person at the other end. And after those grueling 30 minutes, they would understand your situation and do something to rescue you from it. My experiences did not conform to this pattern. It looks like this problem is widespread enough to warrant feature articles in magazines.¹ Indeed, in all the cases that I encountered, the problem was eventually resolved in a miraculous and bizarre fashion; with my interventions seemingly making no impact on the course of things. My experiences were with multiple businesses and in different industries: telecom, banking, and retail. But I think that there are some common trends which can be pinpointed. These trends have made the experience of “reaching out to customer support” in India a painful and exasperating experience. In this, the first of a 2 part series, I briefly recount my experiences. Hopefully, the reader has not been through similar experiences and can read them simply as an amusing recounting of events that will never happen to them.

Telecom

In my experience with the Telecom industry, I wanted to switch plans for a mobile phone connection from Postpaid to Prepaid. This is a routine process and any business worth their salt will be careful to establish a well-defined process which can be followed by all the people working at its brick-and-mortar stores. The business in question was Airtel. Airtel is one of the largest players in the telecom business in India and they have been around for a while. So, I did not expect any issues at all. But despite multiple visits to a physical store, the process was simply not getting done.

The clerks at the store had no clue about what was going on. They were sitting in front of a computer screen and reading out the errors that were being shown to them on the screen. Indeed, their presence was merely aesthetic as they seemed to add neither warmth nor value to the communication I was having with the computer. All of this went up a notch when, in a burst of frustration, I called Airtel’s customer service line 121.

The menu of options is confusing, beyond belief. The main purpose of this menu seems to be to drive the user away from the phone to the app store and to get them to install the “Airtel Thanks” mobile application². Ironically, the reason I was calling the customer service line was because I had already lodged a complaint through the mobile application. However, this complaint seems to have disappeared into the vast ether of the universe because I could not get it in the list of complaints on the same app. I did not receive any other communication about this complaint either.

On the phone, after having listened to endless automated voice guidance read out in a sooth voice that is not at all representative of the listener’s frustration and after delving through some of the options and repeated ventures down the incorrect road, I figured out how to get to a human. The customer service line is not staffed with many people. So, often, there are long waits before you get connected to a human being. This wait generally lasts a few minutes, minutes that one can productively use to contemplate the futility of their efforts in a universe that has been around for billions of years and will continue unchanged for billions more.

At the end of this wait, I was connected to a person whose official designation I believe was “Customer Service Executive Extraordinaire Emeritus.” In typical corporate jargon fashion, this title is meaningless and serves only to divert you from the exceptionally bad service you will soon receive. After explaining what I wanted to achieve to the person on the other end of the line, she told me straight away that there was nothing she could do. The only people who could help were the people at the physical stores. So, her counsel was to simply return to the store. Somehow this conversation would affect the scheme of things and when I next return to the store, the people there would be able to achieve what on 3 prior excursions they had failed miserably at. I wondered if this customer service agent is a believer in a religious sect where doing the same thing over and over again and expecting a different result is the central tenet.

Seeing as she could not help with my original goal, I figured that I might as well lodge a complaint about the store where the clerks seemed to be uninformed about the process and uninterested to check the operation manual, which I am sure they … Well, Airtel probably did not issue them with a operation manual. My naive belief was that Airtel would want to know about the bad experiences their customers were having. What happened right after this convinced me that they did not have any such desire.

I told her that I would like to file a complaint about an Airtel store. She asked me for the address. I told her the address. She was unable to find it. I spelled it out. She was still unable to find it. I sent her a pigeon that had a note tied to its legs with the secret code that she needed to enter into her computer to find the store. I guess the pigeon reached her because she was then able to find it. Alas, this was useless. I am not exaggerating here. It was literally completely useless that she had been able to find the store. To prove my point, here’s the conversation that followed:

Me: Yes, I would like to lodge a complaint about that store.

Customer Service Representative: I see. You will have to visit the store to do that.

M: Um … really?

CSR: Yes

M: Well, that doesn’t make sense. Can you put in a note for someone to check with the store manager and see why the clerks there don’t know the procedure and whether they can learn the new procedure?

CSR: We can’t contact the store directly

M: WHAT?

CSR: We are unable to contact the store directly.

M: What do you mean?

CSR: We are unable to contact the store directly.

… the conversation continued in this vein for a few more lines. Then,

CSR: Well, if you want us to talk to the store manager, then there is an option.

M: Oh, what is it?

CSR: You can go to the store, call 121 from your phone, and connect to a customer service representative. When you are connected, you can ask someone at the store to talk directly with us.

M: Um, okay. Thank you. (I guess?)

So, that’s what happened. In summary, Airtel is a telecommunications business which provides mobile connectivity to hundreds of millions of people. It has stores across the vast landscape of India. These stores have managers and others working there. Airtel has no way of contacting the stores directly. Indeed, there seems to be little or no contact between Airtel and the store, except through the computers that are sitting in front of every single person at the store.

After all this, all I could do was laugh at the absurdity of what I had experienced. I felt much like what I imagine Kierkegaard must have felt like when he said:

As I grew up I opened my eyes and saw the real world and I began to laugh and I have not stopped since.

P.S. We were able to switch the mobile number from Postpaid to Prepaid by visiting a different store and following a procedure that we had heard about more than 3 years ago. The people at this other store also had no idea about this procedure and did not inform us about it or that this was the procedure to follow. Airtel’s consistency is worth appreciating.

Banking

My second experience was with the banking industry. This case was slightly more complex. When my credit card was renewed, I was sent a new one. It was sent to my home address in India. There is no limitation in India that the credit card must be collected by the person whose name is printed on the card. It can be collected by anyone at the household, as long as they are in the correct address. As my family was still living at my home address, I believed that there would be no problems in receiving the credit card. That was a huge mistake. HUGE.

As it turned out, this time around, the card was dispatched from the bank’s central dispatching center. It was sent via courier, which is very common. My home address is in a city and it is very accessible. Delivery people from most courier companies come to my neighborhood daily.

After the first delivery attempt was logged as “Door locked,” at a time that I had already confirmed people were indeed present at the house, I sensed that something fishy was afoot. I wish I could say here that I was able to foresee what the problem was and fix it instantly.

While it is not uncommon for delivery people in India to fill in “Door locked” as a reason for not delivering packages without even visiting the home where they are supposed to deliver it, it is not common enough to be the first thing that one suspects. On further inspection of the scant details I could get from a ghastly tracking page for this courier company (You have not heard about it.), I realized that the contact number on the package was my mobile number; a number on which I was unable to receive calls. My guess was that the delivery person was calling the phone number on the package. On not being able to reach the phone number, they decided that the recipient was definitely not at home and probably dead. So, they understandably filled out “Door locked” as the reason for not delivering the package.

First, I tried to contact the courier company’s customer service. They had no phone line. (Their website had been wiped of any mention of contact or phone numbers by a highly trained team of web developers.) There was a single email ID which they apparently wished all their customers to contact them on. An email to this address elicited a quick response.

Subject: Re: Delivery person did not come to my home (Tracking ID: [REMOVED])

Dear Sidharth,

We sincerely apologize for the inconvenience.We are forwarding this case to the team and assure you of expeditious closure.

Along with this, we have forwarded the request to deliver the shipment on priority and will try to deliver it at the earliest.

In addition to this, we have forwarded the alternative number([REMOVED]) to the team. Meanwhile, we request you to contact the merchant([REMOVED]) to update the contact number.

Best Regards

[Name] Customer Experience [Courier Company]

The content of this email was encouraging. It sounded like someone had read my email, understood my situation, and taken some action. This basic requirement of a “human response” was satisfied! But, as the astute reader might have guessed, this was not to be. My credit card remained undelivered.

The first package was returned to sender after 3 failed attempts due to the spurious reason “Door Locked.” The package was sent out a second time with the same courier company, and the whole dance was repeated one more time. Realizing that my interventions were futile and could serve to only hurt my well being, I decided to lay-off for a period of time and not say anything.

After the package was returned to sender a second time, there were no updates. The bank had contacted me to say that they had been unable to contact me at my address. However, what they really meant was that they had called me and I had not picked up. Curiously, the bank did not include any information about how I might contact them. Seeing as they could not reach me on my phone number, the logical step would be to provide another means of contact. Well, logical to me. To the bank, who knows. Maybe they believe that a person’s identity should be confirmed by only their ability to answer phone calls. Perhaps they have done extensive studies and conducted multiple security audits and concluded that while people engaging in identity theft might be able to physically impersonate another person or enter the victim’s home and receive packages on their behalf, most identity thieves were never able to answer phone calls on behalf of the victim.

In a last ditch attempt, I emailed the general customer support email address of the bank. I told them that the delivery company they were using did not have coverage in my area and that they should use another company. (I might as well try something new as my original story seemed to have no traction with any Customer Service Representative.) This email did not elicit a response. There was an automated response which assigned me an incomprehensibly long “case number.” I wonder why they bother issuing these case numbers. And I wonder why these case numbers are so long. Are they really dealing with billions of customer service cases? Is that not a sign of dysfunction in and of itself?

After a few days, one fine evening, I received an SMS on my contact number saying that the credit card will be delivered to my home in 30 minutes by a different courier company. I had done nothing to trigger this change in courier company. Indeed, about 30 minutes later, the package was delivered. Here again, the bank’s belief in mobile telephony was reinforced! When the person arrives at the door to deliver your package, an SMS is sent to you with a 2-Factor Authentication code. If you are unable to provide this code to the delivery person, you are not you. Of course, it follows that if you are able to provide this code, you must be the intended receiver. *Why believe in government documents that establish your ID when you have a mobile phone capable of receiving text messages?*

Retail

My final set of examples is from retail. It is a well known fact that retail jobs are the worst of the bunch because store clerks have to often deal with adults without emotional regulation, prone to fits at the register, offloading on the poor person standing behind the register. So, my empathy for people in these jobs is quite high. However, the capitalist machinations operate these people have made them do some very strange things.

The first instance was at a cafe inside a mall. After buying something, I went to the register to pay. There the clerk told me that they did not accept cash; that they accepted only credit or debit cards. While the inverse of this is something that most people might have experienced, I had never experienced anything like this. After a few seconds of being surprised by this strange request, I told him that I did not have a card of any sort and that cash was all I had. At this point, another person working at the cafe came to his side, and they conferred briefly. After this, he said that he would be happy to accept cash and opened the register. What do I see inside the register but … paper money?!?! Why did the people who run this cafe start this absurd campaign to accept anything but cash, when their register was clearly stocked and cash is recognized legal tender across the country? I can not say; I can not even imagine a case in which this would make sense. Maybe they like to pay the 3% service charge that companies like Visa and MasterCard charge for the trouble of facilitating credit cards and operating a large network.

The second instance of retail absurdity is really many instances of the same experience. Whenever you buy anything and go to pay for it, the likelihood of being asked for your mobile number is high. I think, in current day metropolitan India, it is basically 100%. Across a wide variety of shops selling clothes, footwear, and food, I was asked for my phone number at almost every turn. Being rather adamant about not sharing my phone number, I have caused lines to form behind me as I try to convince the person at the register that I don’t want another random human being to have to call me and tell me about the availability of money that I will never borrow or insurance that I will never buy. I explain to them how it will waste both of our times and how the activity of entering my phone number into the computer screen wastes the cashier’s time.

This phone number requirement in daily life has reached absurd proportions now. On a recent trip to a mall parking lot, the age-old parking slip had been replaced by … you guessed it … a mobile number. Every vehicle parked inside the parking lot did have a registration number. But once again, the mobile phone seems to have superseded the vehicle’s unique registration number as an identifier for the vehicle! Absurdly enough, the phone number was collected at entry and at exit by a person. It was not used to speed up the process or reduce the number of people who staffed the parking lot. I tried to provide the person with an arbitrary mobile number. This worked, so long as I remembered the mobile number when I exited the parking lot. So, the question I had after the experience was, why change something which gives you neither a monetary benefit nor an improved experience?³ Well, perhaps, customer service absurdity has warped the minds of executives at the top of these companies enough that they don’t even recognize that they are inside a capitalist society anymore.

A cursory search for this problem lead me to this article from 2019 that tries to persuade businesses to use technology to connect customers with humans, rather than with computers. ↩
That is the real name of the app. ↩
I am being frivolous here by suggesting that the collection of mobile numbers does not have monetary benefits. There are reports of companies which build up databases of names and phone numbers and sell them to cold calling firms which will use the database to sell loans / products etc. Surely, the mall earns more from the patronage of a returning customer than from selling a phone number database? ↩

Facebook is Unnecessary

2022-04-29T00:00:00+00:00

Everyone agrees that Facebook makes them feel worse. It accelerates the “Fear Of Missing Out” anxiety that is quite strong even when you don’t know exactly what your peers are doing on a Friday evening. But people continue to flock to Facebook. Yes, Facebook is losing teen users¹, but this is not a precipitous drop. In any case, these teenagers are going to other platforms which are arguably worse than Facebook for reasons that I will outline in this post. What is so interesting on Facebook? Why does it exist? What is it for? These are incredibly complex questions. I will leave that hardest-of-all tasks to the technology philosophers. Instead, I will focus on the one thing that became clear to me after a recent conversation: The existence of Facebook is an anomaly. Facebook’s target user base will never need to use Facebook: Facebook is meant to “connect people.” Their belief is that if you know what is on your friends’ mind, then you are connected to them. The better you know someone, the more connected you are to them. The more you will be able to appreciate the posts they make, both personal and political. But the better you know someone, the less you need a tool such as Facebook to keep that relationship alive and healthy. So, you don’t need Facebook to connect with the people whom you are closest with in the world.² So, …, what gives? Why are you on a platform where you are connecting with a bunch of people whom you know only vaguely and care about at the superficial level? I have some thoughts about this dichotomy.

Types of Posts

Let’s take the premise that posting on Facebook is a right. Broadcasting what is happening in your life is your choice and some people choose to exercise this right.

When you share what you have done or what has happened in your life, I think you can classify the post into one of three buckets. (I know this is a simplification.)

Genuine Excitement: Something has happened in your life that you are genuinely excited about. Excitement is subjective and it is hard to quantitatively judge anything. Why someone is genuinely excited about something is besides the point. If they are sharing something that they are genuinely excited about, then the post falls in this bucket. I posted on Facebook about getting my first internship in 2015. I was excited about that opportunity back then and I believe that I will classify that post into this bucket.
Feeling of Accomplishment: Something has happened in your life that you have been working toward for a long time. You had doubts about whether you would get here, but you have finally achieved whatever goal you were looking for. Once again, subjectivity abounds. What seems a “matter of course” to me, could be the hardest thing for someone else. We all start from different places and subjectivity precludes us from making any judgment about what content falls in this bucket. But the poster must feel this accomplishment and should classify their post into this bucket. I posted on Instagram after graduating from university. It was 5 years of struggling through a program that I did not particularly like and a lot happened during that time, and I wanted to share the accomplishment which I felt.
Bragging: Something has happened in your life and you want to brag about it. Settle down, this is not “Shots fired!” Showing off is just a part of how we operate. It’s not some anomaly which needs to be tamped down on. I disagree that it is a worthwhile pursuit. But that’s my opinion and nothing more. On Facebook, this is simply one of 3 possible buckets that you might classify your post into. On Instagram, things get murkier. There’s not much space to say anything because Instagram shows a small segment of your caption by default. Most of the viewer’s screen is taken up by the high-resolution picture that you have uploaded. I contend that Instagram is a platform where most users would classify most of their posts into the “Bragging” bucket. A few years ago, I bought my first iPod and my first headphones. I posted a picture of the headphones resting on my window sill to Instagram from the iPod. I was bragging about these things that I had now. Looking back at my Instagram, I can see that it was a clumsy attempt at bragging about something, but I can’t bring myself to classify the post into the other two buckets that I have defined here.

Note here that I talk about classifying your own post. So, the onus is on the poster to decide what they are doing on the platform.

Where Does the Frequent Poster Come From?

Experiencing Facebook, one thing was clear to me. It is ~~fun~~ addictive. The number of views on a story, likes on a post, or comments on an announcement continuously increase. The product is designed to draw the user’s attention to these numbers. 3 years ago, I went on a trip and posted a picture from the trip to my Instagram story. I posted rarely enough back then that this triggered a notification to almost everyone who followed me. (The “… has posted for the first time in a long while. Check it out now!” notification.) Within minutes, about 20 people had seen the story. This number kept growing for the next several hours. For the 24 hours that this picture was visible, I could barely stop myself from opening the application, tapping on the story, scrolling down, and checking the list of people who had seen the picture. I have had similar experiences with posting a life update on Facebook, or a comment on a hotly debated post from my friend. The anxiety and excitement of waiting for a response and getting ready to re-engage is exquisite. It feels similar to the adrenaline rush that you would get when you were in a dangerous situation or right before you are pushed down a slide in an amusement park.

My self-regulation was not good enough to prevent this kind of usage of Instagram. I was (thankfully) vigilant enough to realize what was happening and managed to cut out Instagram posting. A few years later, I completely got off Instagram. But most people don’t take this approach.

As the high from a single story becomes duller and duller, more highs are needed. And so, users start posting more. This is how the frequent poster is born: They start out as a standard user of the platform; and then, the platform’s incentives and product design force them into posting more often to get the same feeling as they got from a single post when they started using the platform.

What About Comments and Discussion?

I have seen Facebook posts which fall into each of the three buckets above. And I have seen the comments on those posts. More often than not, the comments section turns into a heated debate about some obscure, tangential subject which is barely even related to the post. It seems to me that the likelihood of this happening is proportional to the abstractness of the topic of the post. The more concrete a post, the less people want to engage with it in the comments section. Why does the comments section feel like a halfhearted addition? Because it was.

As the movie Social Network shows and as early users of Facebook know, the original point of the platform was to simply post updates about your life. The comments section feels like a tacky add-on which is there for the minority of friends who want to say something, as opposed to the majority who want to catch up on others’ life updates. The Facebook mobile app (even now) barely allows you to read all the comments at once in chronological order. It will show you the latest few comments below the post. If you want to see the first comment, you have to begin doom scrolling up to the end of the planet. It seems obvious that the product is designed to keep people scrolling on the feed, rather than go into a single post and check the comments on it.

Why Do People Use These Platforms Then?

Posing big and profound questions like this is easy and fun. Answering them is impossible. I can only point to some of the other things that are happening in society, just as we are starting to notice that people of all ages and backgrounds like these platforms:

Everyone is in a more urban setting now, than a few decades ago. I am extremely glad about this: It is an indication of the economic progress that middle-income countries have made. But urban settings have long been characterized by lax barriers on individual freedoms and cold, disconnected lives lead in a crowd. If you are the type to long for the warmth of a rural setting, then you might wax nostalgic about this. I have no such longings and I think that urbanization is a net good which improves our lives. Nevertheless, the disconnection of an urban life must be balanced by the connection that one pursues on social media.
Organized institutions are declining. This is a phenomenon that has been documented in America. Rotary clubs and organized religion have lost their grip on the population. There are few institutions where people congregate for the sake of congregation. (Developer conferences have become caricatures of the corporate networking events which they tried desperately to not emulate in the early days of the Internet.) Whether this phenomenon is occurring in other countries at the same rate is an open question.
Capitalistic work is inherently alienating. While earlier forms of work and toil involved some pride in the product of the work, most forms of work in capitalism are alienated because the mere worker never gets to see the final product or the financial gains that were achieved through their work. This is not a bad thing: I am glad to not have to “see myself in the products I make.” But what if this was a bad thing? What if the activity that a lot of adults spend most of their time doing must necessarily be something that produces something they feel proud about?

Do You Need Facebook?

We are all at crossroads all the time. We make decisions about which search result to click on and what post to like. What about the Facebook crossroad? Do you need it? I don’t think anyone needs Facebook or Instagram.

Indulge me by letting me state the obvious: These tools exist now and they are not going anywhere. The whole world is not going to stop using them at the drop of a pin. These companies are run expertly by capitalists who have built systems to ensure their survival. What does one person’s mutiny achieve?

It achieves nothing. The refusal to use Facebook or Instagram is an ideological one for me. I think that it is superfluous to my way of life and to the connections that I wish to build with people. If you are not affected by the bragging or if you are friends with people you really care about and don’t mind receiving the same information twice (once directly from a friend and once through Facebook) or you like seeing the periodic heartwarming post that the algorithm serves you or if you use it to catch up on updates from a news service or a film actor or etc etc etc … then it might be a great tool for you. You are the target user. For everyone else, Facebook is unnecessary.

Details about the drop in teenage users in the US emerged in the late 2021 Facebook Files leak. This is a good summary of the documents that were published by the whistleblower and what they indicated about the future of the Facebook platform. ↩
I refuse to use “the real world” here because as much as people from previous generations might try to convince me that Facebook is part of the “virtual world,” I don’t believe such a thing exists. ↩

Monthly Recommendations (February 2022)

2022-02-28T00:00:00+00:00

The month of February is before the end of the financial year and the month that many governments release their budgets for the next financial year. They also talk (at length) about the policies that they are going to implement. To commemorate this annual exercise in long-term decision making, this month’s list of recommendations is about politics and government policy. We see examples of government policy gone wrong and those that worked out well. There is an interesting new theory about conservatism and the feeling of disgust (not just at liberal ideas, but a more generic form of disgust.) And there is an article about degrowth from Vox, which has been making the rounds periodically since 2019. It is worth looking back on this article periodically to understand how futile unilateral action can prove to be.

Liberals and Conservatives React in Wildly Different Ways to Repulsive Pictures (McAuliffe)

Atlantic (archived). Mar. 2019

Political pundits offhandedly talk about liberals, conservatives, left/right, left of center, and so on. They keep stressing that we are all on some kind of spectrum between the extremes, and that our feelings about each subject are different. There is some agreement that political ideologies are sticky and that they are formed based on the household and the environment that one grows up in. There are exceptions, but it seems safe to assume that few people break out of their original political ideologies to make a dramatic shift.

We like to portray ourselves as rational beings who started with a blank slate and ended up at our current political ideology through a process of reasoning. Even conspiracy theorists like to claim that their beliefs are based on evidence and facts. Jon Haidt’s theory says that the apparently rational reasons we have for our political ideologies actually come after we have decided which side to support. Based on this theory, Haidt’s view is that conservatism is “easier to sell.” There is a very strong temptation to believe this. Most people you know are probably conservative on social or economic policy issues, characterized by beliefs such as “Government spending is too high; taxes are too high; ‘culture / politeness / chivalry’ is dead because the current generation does not value ‘relationships / seniority / …’; society is changing too fast.” Haidt’s reasoning is flawless and likely the way things are. Yet, I hold out hope that there is nothing structurally tempting about conservatism and that there is still a way for politicians to sell liberal ideas and have success doing it. So, I found this article about the relationship between conservatism and the feeling of disgust to be a great avenue of research which might help us understand if conservatism is explained by something much more fundamental than our environment: The evolutionary requirement to prevent contamination.
New York’s Real Estate Tax Breaks Are Now a Rich-Kid Loophole (Melby)

Bloomberg Businessweek (archived). Oct. 8, 2021.

Badly done policy is hard to spot. I think the reason for this is that governments write policy documents and they happen to be the only ones with the resources to read the complete document and come to conclusions about its effects. Think tanks have agendas and donors. Polling firms have biases. Also, in most cases, the effect of a policy is not clear until several years after the policy has been implemented. (There are notable exceptions.) Once implemented, the government has little incentive to improve their implementation, irrespective of who is in power. Because if the same party is in power, then it is an admission that their original policy was not the best, an implication that the opposition will pounce upon. And if the opposition is in power, it is an admission that the original policy was good and simply needs to be made better. Both of these are anathema to politicians who want to win the next election. These are the conditions that lead to drawn out political matches, which the media covers with play-by-play coverage and politicians use to improve their standing among their base.

As policies are hard to analyze thoroughly, they will contain loopholes which can be exploited only by large corporations, who have the resources to scour the documents for a thorough understanding. This has lead to a fetishization of “short-and-sweet” policy which doesn’t have as much fine-print. But there is no implicit guarantee that a smaller bill will be devoid of loopholes. This article chronicles one such policy which ended up benefiting exactly the wrong group of people: Children of rich parents with low personal incomes. Other loopholes which have made news in the recent past are related to billionaires evading the estate tax and demonstrate the lengths that the people go to to get out of paying taxes.
‘It’s a Godsend’: 9-Cent Taxi Rides in Rural South Korea (Sang-Hun)

New York Times (archived). Sept. 11, 2021.

Government policy can also bring about radical changes. There are many examples to be found in the Asian economies which caught up to the West in the few decades after the Second World War (Taiwan, South Korea, Japan and now, China). This article about South Korea’s “car-less population rescue policy” is a great example of how this spirit of good government policy remains alive, despite all the opposition that it is facing from a disengaged voterbase and a progressively shortening attention span. Public transport is outsourced to businesses that would like to make a profit on each customer. But when there are very few customers, it becomes hard for governments to incentivize the building and operating of public transport networks. This is happening both Japan and Korea, where the population is rapidly aging and most of the population is concentrated in the urban areas which grew rapidly in the 20th Century. One solution is to make private transport more attractive for rural residents by reducing taxes. Korea has gone in a completely different direction by providing affordable rides from remote locations to the closest towns and bus stops.

This article also shows something very refreshing for people who are living in a bleak policy landscape, where the government’s priorities seem to be completely out of sync with the citizens’ priorities:

When Statistics Korea conducted a nationwide survey in 2010, a lack of public transportation was one of the biggest grievances for older villagers in rural South Korea who had neither cars nor children who could drive for them.

It shows that the government listened to people and their complaints. This, despite being the “way” that democracy is supposed to work traditionally, is becoming increasingly rare.
Anyone Seen Tether’s Billions? (Faux)

Bloomberg Businessweek (archived). Oct. 7, 2021.

Cryptocurrency is weird and it is hard to explain to laypeople and aficionados alike that Bitcoin has become a speculative asset, with no future as a store of value, like a stable currency. The cryptography behind cryptocurrencies is great. When the Bitcoin paper became a big deal in the early 2010s, I read it and re-read it to understand how something that was decentralized could hold any value. That was a pivotal moment for the storage of value. Then, things went south.

Stable coins are an instrument that is nearly the same as a government-backed currency, but mints the benefits of the current “completely decentralized crypto-currency” frenzy. Tether was one of the biggest stablecoins. This article is an investigation to find the cash that is supposed to be backing the stablecoin. It is an interesting piece that delves into the non-conforming characters that were involved in the development of this stable coin (and more generally across the crypto space.)
Can we save the planet by shrinking the economy? (Piper)

Vox (archived). Aug. 3, 2021.

“Modern-day economies have grown too much and the people in them buy too many things / drive too much / eat too much meat / throw away too many clothes / et al. So, to reduce emissions, these people should be told to reduce their activity and return to life as it was a few years / decades / centuries ago.” That is a short summary of the degrowth movement. Activists lobby governments. Governments produce reports. World leaders go to annual climate conferences. Claims are made about having kept the “1.5 degrees Celsius” goal barely alive. I don’t know what that means. Most nations don’t want to do anything except pay lip service and get some “leading the charge” prestige. This is true of corporations as well.

In this environment, a small group of activists suggest degrowth as the solution. Their goals are revolutionary: They think that people will voluntarily give up the luxuries that they have gotten used to. This is a strange thing to imagine and I am certain that it will not happen. But I struggle with the fact that the vegan movement has caught steam and is becoming more and more popular, especially in the advanced economies. Nevertheless, economies must grow and nations, where large parts of the population are still stuck in abject poverty, must reach the standard of living that is common everywhere else. If this growth leads to some warming, so be it. There are gradations of outcomes and we have to choose between a set of trade-offs which ask us how much we value the eradication of poverty against the loss of land and the increase in sea level. I prefer the former. The degrowth movement is an expression of the frustration that some activists have started feeling.

Worth adding to this is David Simon’s counter to the “Less is More” cult:

There are still people, getting up in newsrooms and saying “It’s alright, we are going to do more with less.” No, you do less with less. That’s why they call it less. – David Simon on the Future of Journalism

Notes and Review - Exhalation (Ted Chiang)

2022-02-13T00:00:00+00:00

Ted Chiang is a path breaking philosopher and fiction writer. I started saying that after reading his short story “Story of Your Life.” His second book, “Exhalation”, is a collection of short stories from 2019. It covers an even wider range of topics than his first collection. Chiang has the ability to zoom out of the present moment and write about human nature without providing solutions or trying to pose arguments about complex questions; instead, his writing makes the reader think about what they would do in that situation, and that is the primary method he uses to engage the reader in a discussion. This ability to inhabit someone else’ life for a period of time is the reason I read fiction, and the characters in this collection are put in situations where you want to be riddled by the dilemmas and struggles that they are facing. The striking aspect of Chiang’s short stories is the amount of time that one remembers their premise and key questions for. The key questions in a handful of his short stories have remained with me despite having last read them 3 years ago.

The collection has 9 short stories of varying length. The shortest story is only 4 pages, while the longest one is nearly 100 pages. To review this collection without giving away spoilers, I will focus on the recurring themes and the various ways in which Chiang approaches them.

Capitalism

In “The Lifecycle of Software Objects,” one of the problems faced by the platform that hosts virtual pets is that people don’t spend enough money on it once the initial craze fades. This tanks profits. But the platform still has a small group of “dedicated users” who are always around and very attached to their pets. (Remember IRC?)

Any “rational organization” would simply stop the platform once it stops making a profit. Or they would find some way to make-up for the deficit: Tiered services, advertisements. However, who wants virtual pets that show ads and recommend products to their owners every 3 minutes?

This is the problem that is being faced by many organizations today. Platforms like Twitter, Instagram, Facebook, Goodreads offer free services, which are very useful. But they do not make enough of a profit to justify their own existence. And so, they have been infused with elements that no one likes and that infusion significantly degrades the experience of everyone who uses the platforms. This drives away even those users who think the platform is useful; worsening the doom loop. The network effect has been a key way to keep users using the product despite a deterioration in the UX through advertisements.

Towards the end of the story, the virtual pets have to engage in questionable activities to sustain themselves. This is what platforms are already doing and will continue to have to do in the long-term, if they are survive.

Time

As in his previous collection, Chiang is obsessed with our experience of time and the possibilities that prescience or time travel would introduce¹. Chiang’s writing has been a huge influence in my thinking about anxiety, excitement, and uncertainty, emotions which are byproducts of not being able to time travel and not being prescient.

In the story “The Merchant and the Alchemist’s Gate,” the author poses a question that has been stuck in my head since I read it: *If you could travel in time, would you travel to the past or go into the future? And would you travel within your lifetime or would you go outside of it to find out how things turned out for everyone?*

Needless to say, there is no “right” answer here. Chiang doesn’t expect us or his characters to have a “right” answer. They think about this question and arrive at different answers depending on their position in life. Personally, I have found myself primarily wanting to travel to the future. Sometimes it is to the very end of my own life, to find out what happens and feel less anxious about the present. On other occasions, I don’t want to travel into the future at all, because I am eagerly looking forward to those experiences².

As a bonus, he provides an analogy for loans and credit: You are stealing from your future self.³

Freewill

In the story “What’s Expected Of Us,” the author deals with his obsession over freewill. He does so more directly than he did in his previous collection. The story is a warning and it is incredibly brief: only 4 pages. The brevity of the warning is disorienting and scary because the thing that it warns of has consequences of colossal proportions.

In Chiang’s view, the age-old philosophical question about whether we have freewill or not is not worth thinking about because our belief that we have freewill is essential to our continued survival⁴. It is our sole motivation. If there is no freewill, the story posits that a third of everyone will lose the motivation to do anything and consequently, stop feeding themselves. That is how key our conception of having freewill is, for our continued survival.

Through the certainty that is projected in this story, I felt that Chiang had finally found a resolution for his freewill obsession. I guess we will find out if that is the case when he comes out with his next piece of work.

Conditioning

“What’s Expected Of Us” touches on the effects of losing a conditioning element that we have become used to. Namely, our belief that we have freewill.

“Dacey’s Patent Automatic Nanny” is an amusing story about a little boy who responds only to machines, after being taken care of by a mechanical nanny for a long time.

In the story “Omphalos,” Chiang reassures us that thinking about philosophical question such as “Who created us?”, “What is the meaning of life?” is unnecessary because we can be fulfilled by our own lives and we don’t need to look outside ourselves. However, we are conditioned to look to the majority for approval and breaking out of this shell and looking for fulfillment independently of external input is a method to answer (or, realize that we don’t need to answer) these philosophical questions.

The effect of being conditioned or losing some conditioning is different in each of the three cases, in turn devastating, amusing, and enriching!⁵

Memory

The TV show Black Mirror introduced us⁶ to the disadvantages of having a documented record of everything that we have ever done. What if we could record the accurate truth in one part of the world and continue as usual in another part, and then, compare the experiences of each community? Chiang tries to answer this through the story “The Truth of Fact, The Truth of Feeling.” In it, a tool that can maintain a “life log” is put on a parallel track with the introduction of writing into an oral culture. (What does not exist now but will become available in the future is put on a parallel track with the anthropological knowledge that human beings migrated from being a predominantly oral culture, to one which believed in recording knowledge on parchment.)

The idea is poignant: Writing is a reliable way to document events, whereas stories are unreliable. Both of them have a purpose. Unreliable memory is not inferior to reliable memory. In fact, unreliable memory is good as it allows us to filter out the things that we don’t want to remember. It allows us to tell our story with key milestones that might be misremembered without any malice, because these versions fit better into the larger arc of our life. (A mishap in our past might have put us on the path to a future success. Once we have experienced both the mishap and the success, it is comforting to think of the mishap as a force which was driving us towards the success we did achieve, rather than as an obstacle which prevented us from achieving something else.)

If every experience you ever have is perfectly documented, it becomes impossible to engage in this self-deception. Knowing what happened accurately does not tell you what change it produced in you. It muddles the process of thinking of yourself as someone who has changed, as someone who has overcome hurdles.

This story made me think about my journaling habit. Was it healthy to have a journal which documented contemporaneously the problems that I was facing? I concluded that since I give myself the physical space to write only 3-4 sentences each day and ensure that the journal is not searchable, my unreliable memory is not hindered by this reliable recounting of what happened.

Truth

In the story “The Truth of Fact, The Truth of Feeling,” Chiang theorizes that there are two types of truths: What is right and what is precise. What is right is based on principles, it is not a lie if the right thing doesn’t match what happened in reality, it is simply the way that you experience something yourself. Whereas what is precise is factual, it describes what really happened and must be attested to by witnesses (other than the principals in an event). In cases where the documented record is incomplete, it will be impossible to find out the precise truth. But this should not hamper us from learning from the incident because we will still know what was right.

Parenting

In this collection, the story that I liked the most was “The Lifecycle of Software Objects.” It is a story about virtual pets that fleshes out the far far future.⁷ If you are attached to a virtual pet (i.e. a software object) and then that virtual pet comes to inhabit a humanoid robot, what are your responsibilities towards it? For example, Can you pause the program for a period of time, in an effort to protect your pet from something bad? Or is this unthinkable?

Whatever Chiang says in this story is applicable to parenting. The protagonist struggles with the same kind of dilemma that parents probably struggle with: How do you raise children so that they can be a good person without telling them exactly what to do at each point in life?

Surprisingly, in this story, I saw a side of Chiang’s writing that I have not experienced before. He writes touching dialogue; reminiscent of the things children ask their parents when they are very young.

Later on, after she’s brought the mascots back from the playground to Blue Gamma’s island, Jax tells her about his conversation with Tibo. “Tell him about fun we do time he gone. Tell him about field trip zoo fun fun.” “Was he sad he missed it?” “No he instead argue. He said field trip was mall not zoo. But that trip last month.” “That’s because Tibo was suspended the whole time he’s been gone,” Ana explains, “so he thinks last month’s trip was yesterday.” “I say that,” says Jax, surprising her with his understanding, “but he not believe. He argue until Marco and Lolly too tell him. Then he sad.”

…

“No,” says Jax. He stops and walks back to her avatar. “Don’t want play.” “What? Sure you do.” “No playing. Want job.” Ana laughs. “What? Why do you want to get a job?” “Get money.” She realizes that Jax isn’t happy when he says this; his mood is glum. More seriously, she asks him, “What do you need money for?” “Don’t need. Give you.” “Why do you want to give me money?” “You need,” he says, matter-of-factly. “Did I say I need money? When?” “Last week ask why you play with other digients instead me. You said people pay you play with them. If have money, can pay you. Then you play with me more.” “Oh, Jax.” She’s momentarily at a loss for words. “That’s very sweet of you.”

…

None of the digients was raised on bedtime stories, so text doesn’t fascinate them the way it does human children, but their general curiosity–along with the praise of their owners–motivates them to explore the uses that text can be put to.

– Ted Chiang in “The Lifecycle of Software Objects”

The broken English of the virtual pets here is very similar to the way children talk when they are learning a language. It is the most endearing section of the story.

He even sneaks in a reference to the Alignment Problem faced by people who are building artificial intelligence systems:

You won’t believe what my Natasha did today! We were at the playground, and another digient hurt himself when he fell and was crying. Natasha gave him a hug to make him feel better, and I praised her to high heaven. Next thing I know, she pushes over another digient to make him cry, hugs him, and looks to me for praise!

– Ted Chiang in “The Lifecycle of Software Objects”

Conclusion

This short story collection is well worth the few hours that you would spend reading it, and the several hours after that, which you would spend thinking about the themes and subjects that Chiang broaches.⁸

This is the list of short stories in Exhalation:

The Merchant and the Alchemist’s Gate
Exhalation
What’s Expected of Us
The Lifecycle of Software Objects
Dacey’s Patent Automatic Nanny
The Truth of Fact, the Truth of Feeling
The Great Silence
Omphalos
Anxiety Is the Dizziness of Freedom

In fact, the first story is about this topic and it got me very excited about the rest of the collection. ↩
This is the kind of kooky stuff you will start thinking about once you read this collection too! “Reader, beware!” (For me, “Reader, rejoice!” ↩
Although not completely fleshed out, I think the thinking goes something like this: If you could travel in time, then you would travel into the future, borrow money from your future self, come back to the present, invest in a “sure thing,” and earn the money that you are going to lend to your past self. ↩
Chiang explores similar themes in “Story of Your Life.” These themes were muddled in the movie adaptation, Arrival (2016). ↩
I realized that Chiang gave the collection a symbolic happy ending even in the order in which the effects of conditioning are presented. ↩
Black Mirror episode “The Entire History of You” ↩
I agonized over whether my assertion here that realistic virtual pets, as described in this short story, are not going to come about in the “near future” was a wise one. Being pessimistic about the adoption of new technology and tools though, I am willing to make this assertion. (Despite having the ability to share high quality video feeds when talking with someone, we don’t use it often or for long periods of time due to the unsolved fatigue of video calls. TODO Link post here.) ↩
If you like Chiang’s writing, I would recommend this conversation in which he talks about climate change, fears about AI, and the overwhelming power of capitalism. ↩

Thoughts About The Beatles Documentary

2022-02-11T00:00:00+00:00

I found out about the Beatles documentary from a podcast episode. The host was talking about a clip showing the Beatles legend, Paul McCartney, composing the song “Get Back.” The stunning part of this video was that he had started from nothing. He was idly strumming chords on his guitar, early in the morning. Ringo Starr and George Harrison were sitting across from him; they appeared disheveled, tired, and sleepy. The final Beatle, John Lennon, was nowhere to be seen. I watched this video a few times; hoping to get a glimpse of something around McCartney which gave him the inspiration to come up with the melody. I wonder if other viewers were watching the video looking for a similar kind of revelation. But there was nothing. McCartney had created the song out of nothing; like a vaguely remembered dream converted into a beautiful melody. I learned more about what the documentary was and where the footage had come from. (Admittedly, I went through this information gathering process in a frenzy.) This past week, some theater chains in Japan capitalized on the mania of Beatles fans by airing a 1-hour special, The Rooftop Concert, for a limited period of time. I watched the special, and here are some thoughts about my experience.

Watching the special was a jarring experience: It shows London in the late 1960s in high-definition, color video.

Until now, all of the footage I have seen from before 1990 has been grainy and monochrome. Color movies have been around since the 1940s; but I had not seen any documentary which was shot in color and high resolution until now. The reason I attributed for this is that all of the media that we consume from the 20th century was recorded off a TV screen, possibly originally recorded on tape using the VCR and later digitized. The resolution supported by satellite TV sets back then was much lower than our modern standards of “High-Definition.” So, the only way to get higher quality footage would be to go back to the original tapes with raw footage; digitize those tapes; re-edit the documentary in the same manner; and release the HD digital version of the same documentary. It does not make much economic sense to do this for most documentaries.

For the first couple minutes, we see footage from around London. I was tempted to believe that this was a period movie. This temptation is strong almost throughout the hour-long performance. It is especially strong when either spectators or the view on the street is shown. On the street, people are shown standing around during their lunch hour and listening to the performance on the roof of a building on Savile Row, the street on which the Beatles had a studio where they recorded their albums.

I was so used to seeing the Beatles perform through videos on YouTube that the scenes which included them were the most comforting of the bunch. They perform 6-7 songs on the roof of their building. There is much drama surrounding three police officers who are trying to get them to turn down the racket, owing to noise complaints from other residents and businesses in the vicinity. There are some “taking the crowd’s temperature” interviews happening on the street. Questions are posed to the spectators who are standing around listening.

The hour-long special used IMAX technology, which meant a larger than normal screen and particularly sharp, surround-sound audio. This made the whole experience a lot of fun. I had resigned myself to being able to listen to bands from the 20th century only on domestic audio equipment. The full version of this documentary, about 8 hours long, has already been released on Disney+ (or Hotstar, if you are in India). So, I caught myself thinking about whether it was worth going to the theater to watch something that was already available to me online, on demand. However, just as quickly, I realized that I was comparing two completely different experiences. (Irrespective of how good or bad a movie is, I have never regretted watching it in the theater. So, your mileage might vary.)

I was taken aback by how familiar London looked. I have never been there myself, and all my prior knowledge of it has been through either news footage or TV shows set in Britain. Particularly familiar was the attire of everyone on the street. Even the cars that were on the street and mundane details like the glasses that people were wearing: Everything just seemed familiar and period appropriate.

I could not explain this familiarity, or my heightened sensitivity to it, because my understanding was that fashion has changed significantly since the 1960s. One possible explanation is that our expectations of each period have been conditioned by frighteningly accurate portrayals. This conditioning has made real historical footage neither surprising nor disorienting, as they simply serve to confirm one’s prior expectations of the period being depicted.

The questions on the streets brought out a very interesting phenomenon, to whose existence I had not paid the requisite attention. The Beatles were not universally liked. They were very popular, but many people thought that their music was too similar to pop music; too simple; “very indulgent,” as Bob Dylan quipped. Liking the Beatles was a slightly rebellious act in the 60s, I presume. Only slightly because their lyrics were wholesome and there was little of disrepute or “Satanic influence” there that someone could reasonably complain about. Indeed, two adults say that they have not heard the Beatles themselves, but that their children buy their records. Liking pop music, ironically, is always frowned upon by someone. There were two elderly people who were unhappy about the racket. One of them complains about having been woken up from her sleep. That, I thought, was a funny touch.

The drama with the policemen leaves one feeling conflicted. They were doing their job in trying to stop the performance. When the policemen reach the roof and are standing behind the band, as the band is performing, McCartney turns back and looks at them multiple times. His demeanor clearly shifts after the first time he looks back and notices that the police are trying to shut them down. This renewed perception of the performance as an act of rebellion seems to energize him. He starts smiling, and is clearly willing to taunt and defy the policemen, should it come to that. As their last song, Get Back, comes to an end, he adds a few impromptu lines about how the police don’t like them performing on the roof and might even arrest them for it.

I have one final thought about the 1-hour special: The questions that were asked on the street were cringe worthy. The conversation always seems to run something like this:

Question: Do you know what you are hearing?
Answer: It is the Beatles, right?
Question: Yes. Do you like it? / Would you buy their records?
Answer: Yes, absolutely. I love them!

Or, like this (probably worse):

Question: Do you know what you are hearing?
Answer: No
Question: It is the Beatles
Answer: Oh, okay
Question: Do you like the Beatles? / Would you buy their records?
Answer: Sure, I will try it out

Every time the interviewer said “The Beatles,” I cringed. I think that this idea to ask people on the street about the performance was the documentary producers’. Perhaps, they wanted to show that the band was well-liked, and that this stunt would pay-off big time. However, for a band as popular and well-known in the 1960s as them, I thought that it was superfluous to repeat this conversation several times and get people to approve of their music. At times, it felt like pandering to a future audience; an indirect way of saying “The Beatles were a big deal back then. We knew that. You should know that too, viewer in the future.” I did not get any kind of wholesome message. The producers could have been secure in the Beatles’ position as a genre-changing musical force.

Monthly Recommendations (January 2022)

2022-01-31T00:00:00+00:00

For the first recommendations post of the year, I have picked out some of the best New Yorker stories that I read as I worked through my issue backlog from early 2020. I have given up on reading the New Yorker at the pace at which it is published. However, the articles that are included in each issue are rarely “current affairs” related and can basically be read and re-read years into the future. Lately, the role played by magazines in our reading diets has become clearer to me: They are published at a much faster pace than great fiction or non-fiction; while they are written by the very same people who will eventually produce those great works; so magazine pieces offer us a window into upcoming great work. (I think.) So, this month I have a mix of dealing with grief, bad governance as seen through the eyes of an official, compared with the impact of bad policy as seen through the eyes of a journalist, the “mad science”-y feel of triggering avalanches intentionally to avoid larger avalanches, and a disturbing article about the effects of late-stage capitalism on workers.

On The Warpath: A profile of John Bolton (Filkins)

New Yorker (archived). 6 May, 2019.

How do the people inside government really work? Several meetings with people who have radically different view on subjects; an exchange of opinions which will most probably end with no one changing their mind; access to decision makers and the people in charge being closely controlled. Most of this portrayed on TV shows about the government: Veep, about the peculiar American system; and Yes, Prime Minister, about the more common Parliamentary system. This profile is another confirmation that this is really how governments work in reality too. Bolton was one of the people in Trump’s cast of national security advisors. Bolton’s main feature was that he was hawkish, which is a polite way of saying “war-mongerer,” as far as I can tell. His appointment seems to have simply fueled the lone wolf mentality that was prevalent in the Trump administration right from the beginning. This quote from an official under him is priceless:

Under Bolton, there are fewer meetings, less collaboration; he often disappears into his office to immerse himself in documents. … “He reads the memos. There just isn’t a lot of feedback.” Some former officials believe that Bolton’s insularity could be dangerous, particularly in a crisis, when various arms of the government and the military have to mount a quick and coordinated response. “It’s chaos under Bolton,” the former senior national-security official told me. “The national-security adviser is supposed to facilitate the President’s directives and coordinate national policy among the various government agencies. That process has completely broken down.” The official added, “Bolton hasn’t set any priorities. No one knows what the policies are–what’s important, what’s less important. …”

Principals’ meetings–crucial gatherings involving the President, the Joint Chiefs of Staff, and the heads of intelligence agencies–have become rare. “I don’t remember the last time there was a fucking principals’ meeting,” the official said.

I don’t think anyone wants to hear that about their government’s functioning.
Snow Science Against The Avalanche (Somers)

New Yorker (archived). 23 March, 2020.

Science can be used to study practically anything because the application of the scientific method is not constrained by the field that one is applying it to. This article is about the application of the method to the prediction of natural disasters. Avalanches are fascinating because it’s hard to imagine how they begin or how they can get as terribly big as they generally do. They also happen to form almost entirely mysteriously with everything happening below the surface with precious little to monitor. This article is about the patrollers whose job it is to forecast avalanches (in an effort to protect skiers) and to trigger small avalanches which will not cause any harm to avoid the large avalanches which are generally devastating. There is a lot of science behind all this including scales to measure the risk of an avalanche and the mechanics of accumulation of snow on a slope. This article was a fun read. (I don’t do any winter sports, so it was mostly just fascinating that this kind of rigor existed in what I had considered to be basically a random event which no one could predict.)
Why Americans are Dying from Despair (Gawande)

New Yorker (archived). 23 March, 2020.

This is an article foreshadowing an unusual and scary phenomenon that will occur in late-stage capitalism. As businesses try to reduce their costs progressively, almost everything will be automated. While there is widespread debate about how much will be automated, it is certainly more than the state we are in right now. It follows that large groups of people will probably lose jobs in the future. Ezra Klein has repeated in multiple podcast episodes of The Ezra Klein Show that the loss of people’s jobs is not just a problem which economists need to deal with, it is problem that society has to think about before the coming wave of job losses. The reason for this is that the jobs that people are involved in are more than just a source for money: It is the source of dignity for workers. When this source is suddenly taken away from them, they’ll face the kind of disillusionment that leads to societal collapse. Most countries are not thinking about this problem because it still seems far off. Whether this problem will even occur is up for debate because Japan is facing a severe shortage of labor because of an aging population and a dropping birth rate. The dropping birth rate seems to be common and unavoidable as even China, the latest to enter the late-stage of its development, is facing an identical problem. Nevertheless, how will economies that are putting their eggs in the capitalism basket deal with the problem?
Abandoned (America’s Abandonment of Syria) (Mogelson)

New Yorker (archived). 27 April, 2020.

As the withdrawal of America from Afghanistan played out over the last few weeks of August 2021, two prominent, prior American withdrawals came to mind: America’s partial withdrawal from Afghanistan in 2003, to focus on the war effort in Iraq; and America’s subsequent withdrawal from Iraq about 8 years after the overthrow of Iraq’s leader Saddam. Both these withdrawals ended up relegating the countries to various states of ruin. Afghanistan’s new Taliban leadership seems committed to some form of governance that doesn’t allow regional militias to have undue influence over the general public. Iraq’s government barely saved the country from becoming a failed state, and is in the painful process of recovery; especially because it is unable to provide even the basic necessities of life to its people. America’s withdrawal from Syria was orchestrated by Trump as part of his policy to end long-running wars and it was an equally large catastrophe. This article focuses on the confusion within the American military hierarchy and the crucial misunderstandings between Trump, who was in charge of policy, and the rank-and-file generals, who were in charge of communicating upcoming policy decisions to their Syrian allies. The article highlights the damage that was done to Syria’s future. It is unclear who is to blame for this misunderstanding; even if the blame could be pinned on someone, it would be futile.
Missed Calls (Collins)

New Yorker (archived). 11 May, 2020.

Over the past few years, I have stopped using Instagram and Facebook. This has led to no communication with people who are exclusively on one of those 2 platforms. I have known (subconsciously) that my refusal to use these services for reasons that involve my well being or my ideology is a selfish and subversive refusal to stay connected with these people. This article was an uncomfortable account of what it feels like to be on the other side of this divide. It also made me think more about how drastically our expectations about communication have changed in a relatively short period of time. “Stay in touch” was a corporate euphemism for “Remember that we worked together because it might be mutually beneficial in the future.” Now, this phrase has become commonplace even among friends and family.

Video Calls Are Tiring

2022-01-29T00:00:00+00:00

Video calls are ubiquitous. Skype and its modern counterparts were the go-to tool for connecting people who were not in the same physical space. In the era when travel was possible, convenient, and exciting¹, these tools were stopgap solutions; to be employed until the time that you could refill your “physical presence” account balance with people. Then, COVID19. Everyone was stuck at home. Travel became impossible. Video calls became the only way to meet some people. At work, the number of people that I interact with on a daily basis whom I have never met in real life has gone up from 0 in March 2020 to 5 this past week. I don’t think my experience is an outlier. The necessity of using this tool could not be escaped; neither could the feeling of tiredness that would always follow its use. What is the source of this tiredness?

The fundamental characteristics of a video call have remained depressingly constant since the initial days of Skype: a small rectangle with your face and a large rectangle with the other person’s face. Consider this quote from a 2011 article in the New York Times:

But the disconcerting thing, he said – as others have mentioned – is that the current technology means one is rarely looking eye to eye. “When someone doesn’t look us in the eye, the brain deduces this as someone being less likable, less confident and less honest” – https://www.nytimes.com/2011/05/21/your-money/21shortcuts.html

As it happens, the “current technology” of today, 2022, still does not let you look eye-to-eye. To exacerbate the problem, there has been a wild expansion of the primary use-case, “establishing business relationships,” which was being considered in 2011.

Video calls replaced every activity that called for congregation: work, school, happy hour, dance classes. They are a poor replacement for congregation; no one is on that side of the debate anymore. Conversations about video calls replacing in-person meetings were in vogue in 2011, and they were in vogue early in the pandemic, when work-from-home was reducing work travel and giving an unprecedented amount of flexibility to employees.

Adults might be surprisingly good at using these tools for work. Children are not doing well using them for school. I will repeat the oft-repeated truism: “Video calls lack the human touch.” Every uttered sentence is quickly transformed into a cold, emotionless gob of words, devoid of facial expressions and hand gestures, making the communication of a simply sentiment tortuously hard.

It was the best worst solution for the problem that faced almost everyone. But people in jobs that involved meeting people for 8 hours a day noticed early on that they were tired after meeting a lot of people online. And more so than they would have been had they been meeting the same people in person. I felt the same way during the workdays when I had meetings for several hours at a stretch.²

The initial explanation of this tired feeling was external to the primary communication tool. The blamed was pinned on other plausible sources of anxiety: the news cycle was crazy and we were bombarded with depressing information (China, Italy, New York, …); spaces at home were not designed to be conducive to a comfortable work day anyway; we just did not have the right equipment, and other arguments in this vein. This went on for some time. However, these explanations were not enough. People were tired and the reasons had nothing to do with external factors. They started looking within and at the tools that they had taken for granted until then.

There was soon an explanation that involved performance anxiety: online performers and live streamers who had been broadcasting for a long time said that all video calls are performances and everyone is on a stage. Moreover, this online stage is worse than a physical stage because it is equipped with a mirror which shows you what you look like.³

This explanation made sense to me. Even though I did not think of video calls as a performance, I saw that I was transfixed by the small rectangle that I was broadcasting. This small rectangle had no new information; I should not have been looking at it at all. Nevertheless, it was the natural resting place for my vision on the screen and I kept returning to it every few seconds.

To this explanation of performance anxiety, I want to add 2 supplementary points. I believe that these exacerbate our anxiety and self-consciousness, and make our experience of video calls unnatural and tiring.

Strange Proportions

First, the unnatural size of the opposite person. My phone screen is 5 inches tall and 3 inches wide. The average human face is approximately 10 inches tall and 7 inches wide. To deal with this problematic difference in proportions, wide-angle cameras take to cropping the feed appropriately. This gives rise to a mash-up of strange proportions.

If the speaker is sitting too far off, a small face surrounded by empty space. And if the speaker is holding the phone too close, an impossibly large face that takes up the whole screen. There is no way to win. I noticed that I always ended up with a video feed which is great for the first few minutes of a conversation when I was assessing it with curiosity. As the conversation continued and I focused on what the speaker was saying rather than where they were or what they were up to, the strange proportions became unavoidable.

There are recommendations to fix this: “Keep the camera fixed. It should line up with your line of sight. Sit in a chair at a comfortable distance from the screen.” This advice is easy to follow when you are at work inside your house. But when you are on a video call with family, it is hard to stay in a single place and maintain unchanged posture for long periods at a stretch.

Expectation to Not Look Away from the Screen

Looking away from the screen during a video call is taboo. When meeting someone for the first time at work, I will tell them that I am looking away from the camera because I am writing on a notebook in front of me and not because I am being rude. The expectation to not look away is strong. It is a sign of politeness online, where the other person can’t see what you are staring at off-camera. My assumption is that the other person will assume the worst thing about an innocuous off-camera look: namely, that I am reading a tabloid article on my phone!

In real life, we rarely stare at another person’s face for a long period of time. Even when two people are talking, there are often things around them and looking at these things for a brief moment or moving your head around is a natural way to break up the monotony of an unchanging physical stance.

This realization came to me through Japanese lessons. When the lessons were conducted in real-life, I rarely looked at my instructor’s face when I was working on assignments or answering questions. I would spend most of the time looking at blank walls, trying to come up with the vocabulary or the grammar for describing a particular situation.

When the lessons shifted online, there was no expectation (either my own or my instructor’s) that I would stare at the screen/camera because I had never done that before. During these online lessons, I would often stare outside a window to my left or at the wall to my right. These 1-hour lessons were not as tiring as similar one-on-one meetings at work.

Eventually, unbeknownst to myself, I started applying the same tactics during my one-on-one meetings too. Through this exercise, I consciously weakened the expectation that I wouldn’t look away. Looking at your surroundings is a part of being natural offline. There is no reason there should be any expectation that we are stiff and monotonous online.

Video conferencing software will often put the “shared screen” of the other person front-and-center and hide the person’s face in a sidebar. Google Meet used to do this by default before. If both people are staring at the shared document, they are not staring at each other’s faces. This is a good start, however it is not a sufficient solution because the underlying expectation about not looking away from the screen is just as strong as before.

Conclusion

This tiredness was coined early in the pandemic: Zoom fatigue. Technology philosopher L M Sacasas theorizes that Zoom fatigue is related to the way we perceive each other’s bodies on a video call. A video call pretends to be equivalent to a physical meeting, despite being a terrible imitation: there is neither the comforting, shared physical space as in real-life conversations, or the intentional disembodiment of the telephone. The physical cues that our evolutionary history has trained us to look for are expected, but missing. Video calls are good for communication, but they are decidedly bad for people.⁴

Zoom fatigue does not have a simple solution. One solution is to stick to the “intentional disembodiment of the telephone.” A better solution is to start by sharing video and subsequently, switching to only audio. (It is hard to get the timing right.) The solution that I prefer is to cut through the pretense that video calls are “real-life conversations with caveats.” If we start thinking about them as “telephone conversations with a few perks,” video calls will be less tiring and more enjoyable.

… and not ridden with the anxiety of getting an RT-PCR test result in a foreign country. ↩
Being at a company where I was not required to turn on my camera for all meetings, I resorted to turning my camera off and pacing around the room whenever I could. ↩
Google Meet implemented a feature which would fix this problem: It allowed people to hide the video feed that they were streaming. This feature is not persistent, i.e. you have to hide your feed in every video call, which can get tiring in itself. Other applications such as WhatsApp and Facetime which are more commonly used in family settings don’t have any such feature yet. ↩
Paraphrased from Sacasas’ blog post about Zoom Fatigue ↩

Nostalgia vs. Anxiety on Social Media

2022-01-16T00:00:00+00:00

When Facebook came on the scene, the most attractive features on the platform for me were photo albums and life events. These two features in combination gave the user the ability to build a timeline of their life. Every trip that you go on, place that you visit, and dinner that you eat can be documented for the present and archived for the future. The photos and events were arranged as milestones that the user chose to retain when they told their story. The digitized nature of this data enabled the creation of works that were out of reach for the ordinary scrapbooker or the amateur video editor: The personalized Facebook “Lookback” videos, which were generated for Facebook’s 10th anniversary, were the biggest “proof-of-concept” of a world which was digitizing at a fast pace. I feel that Facebook was at “peak utility” back then. Nostalgia was the currency that the platform traded in effortlessly. Product managers at Facebook intuitively understood the value that they were creating in their users’ lives. Looking at the Lookback video is not like looking back at photo albums from a decade ago because in the latter experience there is no curation. However, Facebook’s usage is on the decline. Platforms that focus on creating Anxiety in their users’ minds are on the rise. Did something fundamental change?

I really liked my personalized Lookback video released in 2014 and I must have watched it at least a dozen times, noticing new details every time I re-watched it. Facebook used technology to build something that could not have been built without it; they produced something for everyone which was, until then, out of reach for amateurs. I can say this without any qualification: Facebook was at its peak utility at that point in time. I wonder if I would have continued to use Facebook if it had stood out as the only platform that focused on nostalgia, among a gaggle of platforms that focused on making people anxious by showing them what everyone was else doing right now; a wave of products that ostensibly began with Snapchat Stories.

The most popular platforms of today are good at sharing “the current moment.” They deal in the currency of Anxiety. They force their users to share the present, which fuels an urgent need to do something interesting. Why did social media become a place where archiving lost its value and sharing the present with a large audience was the primary objective?

The popular platforms of this new era are Tik Tok, Instagram, and Snapchat. Tik Tok is the only platform that tries to get its users to produce content that is widely shared and holds for a relatively long period of time¹. Instagram and Snapchat are completely invested in the “right now.” Their investment in the “now” is different from the historical goals of Facebook. On Facebook, you share what’s happening now for the benefit of the future. When you create a Life Event on Facebook, that event stays there forever, just as the event changes you, the event changes your Facebook timeline. When you add a photo to your Instagram Story, it disappears in 24 hours. Nothing you do on these platforms has any lasting impact anymore. Archiving has become a thing of the past. Even in the cases where content goes viral, the phenomenon has a dizzyingly short lifespans².

There is no question about the currency that is valued higher in capitalism. Nostalgia has its place in society, but Anxiety is where the money is at. When things are going to disappear in 24 hours, you probably want to consume them right away, procrastinating for a few hours. A newspaper that publishes every 10 minutes will have significantly higher turnover than one that publishes every day, as long as people are interested in the content and the content producers can work at that rate. With everyone signing up to produce and consume content, both those conditions are met in the “push notification newspaper” world.

Anxiety (in my opinion) is the manifestation of scarcity that is sensed in a system. There are limited seats in a movie theater and if you really want to watch a movie, watch it as soon as you can and book the ticket as soon as they are available. Artificial scarcity marketing techniques work well for brick-and-mortar businesses like Apple (which used to have cash sales) and even publishing houses that used to release books that people would wait for several hours to get their hands on. (Rowling’s Harry Potter series comes to mind.) “Don’t put things off because you might lose the opportunity to ever experience them.” This insight into the effect that enforcing scarcity would have on their users’ minds was the driving principle behind the product that started this modern wave of platforms: Snapchat Stories.³

Archiving the past has been delegated to other platforms like Google Photos and Apple’s iCloud. These services also attempt to create artsy videos using your pictures, grouping them by location and the people that you are with. The data that Google or Apple have about your pictures is an insignificant subset of the data that Facebook had about the same pictures. These auto-generated videos can’t be compared to what Facebook created.

Facebook had the ability to figure out what was important to you. Facebook knew about your most liked post. The more life-changing your announcement, the more people are going to engage with it; the more people will congratulate you, and so on⁴. Apple and Google have no idea about the importance of any life event or even about the presence of a life event behind a set of pictures, that were taken in close spacial or temporal proximity. The effects of this ignorance are clear when one sees iCloud’s attempts to build videos. Every video looks the same. They are templates rendered with different photos. There’s no personalization. There’s no understanding of how important a set of pictures was to the user. Even the name of the auto-generated video is just an amalgamation of the month that pictures were taken in and the places where the pictures were taken.

Will the value of archiving rise again? After all, the Sepia tone filter is a mainstay on most Camera applications today and people still post old pictures to Instagram Stories on Mothers’ Day.

I don’t think archiving will make a significant comeback. Human nature has not changed. I will not argue that point.

But the thought process behind nostalgia has changed as people understood how to use the online tools that were at their disposal. Online tools prefer things that can be safely publicized. The ability to share content is key for the creator of successful⁵ online content. Private experiences have their own space, but those spaces are an afterthought in modern platforms. Experiences that create Nostalgia are private. Experiences that create Anxiety can be publicized. A Lookback video is interesting to the subject and the handful of people who are closest to them. A Snapchat highlight about their recent trip to Europe is mildly interesting to almost everyone.

A new platform might come around which tries to work with nostalgia and convince its users that their feelings about the past are more important than others’ admiration of the present. Personally, I am pessimistic about the return of nostalgia because the success of modern platforms has convinced me that we continue to disregard Seneca’s advice from over 2000 years ago.

Don’t take pleasure in the majority’s approval.⁶

Everything is relative to the average attention span of the modern user which is at a dismal low. ↩
Some trends evade explanation with their absurdity. ↩
Ironically, Snapchat lost to Instagram because it could not address the scarcity problem that it was facing: the scarcity of users. ↩
I am putting this here as a historical view of the platform’s operation. I haven’t used the platform lately, so my views about the posts that gain likes and engagement might be outdated. However, I believe that the basic assumption that more people will congratulate you the larger the importance of an event still holds true, irrespective of how behavior might have changed online. ↩
Success has different yardsticks of measurement. Here, I will simply equate success (in the mind of the content creator) to the number of likes that a post eventually gets. ↩
Letter 7. Letters From A Stoic (Seneca) ↩

Monthly Recommendations (December 2021)

2021-12-31T00:00:00+00:00

There is no specific theme for this month’s recommendation list. It is an assortment of the articles I read over the past year which had one thing in common: I wanted more people to read them and decide what they thought about the underlying institutions and people that these articles described. So, we have two articles about the tennis world: one about an athlete within the world and another about the incentive structure of the sport; one article about a non-contrarian CEO with some interesting ideas about running companies that he does not divulge, and another one about a politician with new realizations that helped him confirm 3 nominees to the US Supreme Court even though the confirmation of a single Supreme Court nominee was hailed as a big achievement just a few years ago on the TV Show, West Wing.

Naomi Osaka and tennis journalism’s ugly history of demeaning its players (Abad-Santos)

Vox (archived). 17 June, 2021.

Naomi Osaka decided to take a break from tennis after she was fined for refusing to appear at post-match press conferences at the French Open. The moment she announced this was included in Google’s Year In Search 2021 video. It was a major event in tennis; it was a major event in sports in general: A player decided to stop playing the sport because they could not deal with pressure from the journalists who cover that sport.

I don’t agree with the viewpoint of this article from Vox. Vox has been noticeably moving to the liberal left in the past year, especially since the departure of the two founders and more of the founding staff. This article seems to be another indication of this. Sports has never been just about the game that is being played. It has always been about the human element: People play these games and it is these people we are interested in, more than anything else. This was important for me in understanding the disgusting behavior of the spectators at the US Open 2018 final between Naomi Osaka and Serena Williams who were booing the victor. If it was all about the sport, people would not be booing anyone because they got to see a good fight.

If the athlete decides to just leave and not talk to anyone until they have had the chance to re-watch their performance and talk to their coaches/PR consultants to decide what they are going to say, the apparent “raw” element of sports is lost. This shift might not be a bad thing, but if this really does happen, the group of spectators will also change.
A Few Tennis Pros Make a Fortune. Most Barely Scrape By. (Steinberger)

New York Times (archived). 29 June, 2021.

The incentive structure inside the professional tennis world is messed up. There are multiple tiers of competitions in which professionals early in their career must compete and win to get spots in the noticeable tournaments. These early tournaments are tiring and often played without any sponsorship. A player that does manage to travel and compete in these early tournaments will face the daunting task of defeating a well-rested seeded player in the more popular tournaments. This system is beneficial to players who have already made it and perhaps answers the question about why the established players from 10 years ago are still the established players of today (Nadal, Federer, Williams). The New Yorker published a fictional take of the same problem, showing the state of a player who depends on benefactors to pay for his practice and travel, and develops a relationship with the donor’s offspring.
These Chinese Millenials Are ‘Chilling,’ and Beijing Isn’t Happy (Chen)

New York Times (archived). 3 July, 2021.

Opting out of the rat race is not just a personal choice in China. It is an interesting problem for a government to solve. Governments have been working to increase the workforce participation rate for several decades. But if they work their population too hard, the efforts are bound to boomerang and blow up in the government’s faces. This is a good indication of what late-stage looks like in the socialism/capitalism hybrid that the Chinese economy is based on¹. We have already seen what late-stage socialism looks like in Europe with the pro-status-quo, anti-immigrant sentiment that has been growing over the past few years. In America, late-stage capitalism has manifested as dizzying inequality between executives and workers. India’s capitalism-based economy seems to be heading towards the inequality scenario. Looking to other economies, identifying parallels, and trying to prevent them is a worthwhile exercise that policymakers seem uninterested to engage themselves in.
Does Palantir See Too Much? (Steinberger)

New York Times (archived). 21 Oct, 2020.

The present is the time of the contrarian CEO. Just look at Elon Musk and his cavalier usage of his Twitter account to poke fun at regulators despite being burned before. This article is a profile of Palantir’s founder Alex Karp. Karp is a contrarian CEO, who thrives on the mystery that he is able to generate. He is also non-conforming in the way that he runs the company. He talks about how the “team” is more important than everything else; which is similar to a lot of other software companies and not contrarian at all. But one of the advantages of his “let us maintain the mystery about Palantir” routine is that it makes it impossible to tell whether he is running the company in the standard fashion or if he is doing something truly contrarian. This article is a fun read simply due to the eccentric nature of the CEO. (Take for instance, his “inability to be norm-conforming and marry.” Or his flight to Germany to study German and read the writers and thinkers he was drawn to in their mother tongue.) The way that Karp talks about himself and Palantir in this article makes it seem like a cult. This might be the very effect that Karp is going for.
Enabler-in-Chief (Mayer)

New Yorker (archived). 20 April, 2020.

The American political system is ridden with veto points which can be exercised by a single person. This also means that a single person who is able to keep their party’s policymakers together in an unbreakable coalition is extremely valuable. Former Senate majority leader Mitch McConnell from the Republican party was just such a man. He was able to keep all the Senators from his party united towards a single goal, a staggering number of times during his 5-year term as the majority leader (since 2015). The one time that he had some doubt about whether he would be able to keep his caucus together was when Obama nominated a Supreme Court justice in his election year. On this occasion, McConnell decided to keep this from even being voted on. Even during the last few chaotic weeks of the Trump presidency, McConnell managed to confirm 1 Supreme Court justice and prevent the impeachment in the “second impeachment trial” of Trump after (and despite) the events of January 6, 2021. His understanding of the rules and procedures of the Senate were unparalleled. His realization that the electorate will blame the party in power for losses, even if they were caused due to a filibuster from the minority was genius and the key to most policy that did not go through the American system whenever the president is a Democrat. This article delves into the “McConnell” psyche by talking to people who worked alongside him and got a chance to look at how he makes decisions. The most memorable quote about McConnell, unfortunately, is one that shows a total lack of policy-making principles:

He [McConnell] never had any core principles. He just wants to be something. He doesn’t want to do anything.

“Common prosperity” is the CCP’s answer to the widening economic inequality. Some policies that are included in this such as limits on for-profit education institutions are worth adopting in nearly every economy that wants to improve education opportunities for their citizenry. — ↩

Notes and Review - Meditations on First Philosophy (Descartes)

2021-12-26T00:00:00+00:00

Descartes has a gift for reasoning that I will not attempt to summarize in the few paragraphs of this review. Descartes’ essay is worth reading if you have ever wondered about the philosophy behind his saying “I think, therefore I am.” It is worth reading if you have a few hours to yourself and you want to think. It is challenging, and some of the philosophy went over my head. Despite that, I got a basic understanding of what he was trying to do. He attempts to reason from scratch to prove the existence of the “soul” and of “God,” for he believes that if he is able to prove their existence, then even the most irreligious person would follow his reasoning and become religious. I was not completely convinced by his logic in this regard, with regard to his proof of God’s existence. But I was convinced about the reasoning behind his quote connecting doubting, thinking and existence¹. In this review, I have explained both Descartes’ lines of reasoning and the doubts that arose within me when I followed these lines “without prejudice”².

Descartes’ Reasoning

Doubt and Denial

First off, Descartes tells us to do something apparently elementary: Deny anything that might have even a shadow of doubt associated with it.

For e.g., Is your body your own? Let’s go down this rabbit hole just as Descartes tells us to:

When you’re asleep, you might dream yourself to be inhabiting another body
When you’re dreaming, the dream appears as real to you as reality does when you’re awake
Your mind thinks that your body is your own
Your mind thinks that the body you inhabit when you dream is also your own
Another mind could think that your body is its own when that mind is dreaming
Hence, your body might belong to another mind!

There! That’s a shadow of doubt. So, deny the original question:

Q: Is your body your own?

A: No; for there is some doubt whether my mind is the only thing that owns it.

This sounds ridiculous; but according to Descartes, this is how philosophy should be done. We must start with nothing and reason to something.

My Mind Exists. I Exist.

With the above example, it is clear that your body is not your own. What about your mind? Here, Descartes makes his first major claim. Your mind is your own as it produces thoughts. A thought can exist separate from the mind that produced it. And thus, that is what I am: I am a thinking thing, with a mind of my own, that I use to produce thoughts.

I know what I am. But what about my existence? For it is quite obvious, that I could be something but not exist. (E.g.: I know what a building with 1000 floors is. But that building does not exist. So, I should prove the existence of that building separately.)

So, do I exist? Yes! Rejoice! I know that I exist because it was I who thought and judged my existence. To produce a thought, there must be a thinking thing that produced it and that thinking thing is me.

God Exists

Let us accept the following axioms³:

A thing that is not perfect can’t produce a thing that is perfect
Something cannot come into being from nothing

Now, look into yourself, and follow the following line of reasoning:

I have the idea of God: a perfect, infinite being
I am not perfect
I am finite
Thus, my mind, a thinking thing, can’t produce the thought of a perfect and infinite being for it is neither perfect nor infinite
Hence, this thought must have been placed into my mind by a being that is perfect and infinite
I choose to call this being God
God Exists!

Well, there you have it. That’s the line Descartes takes to prove his hypothesis.

My Doubts

As I said in the introduction, I was not convinced by the author’s reasoning. Here are some of the doubts that entered my head as I read this essay.

The Nature of Thoughts

Let me go back to something that a reader might question:

To produce a thought, there must be a thinking thing that produced it and that thinking thing is me.

Elsewhere, Descartes says the following:

Something cannot come into being from nothing.

I said before that Descartes wants to start from scratch. And yet, when I made the above statement to explain Descartes’ reasoning and when Descartes said something similar, I am accepting that to be true. Where did this “thought” come from? Or, more simply,

Who produced the thought “a thought can’t exist until a thinking thing produces it” in my mind?

This is the axiom³ that Descartes relies on.

God’s Existence Hinges On A Definition

In the above reasoning the whole argument hinges on the fact that the thought inside us about God is that He is perfect and infinite.

What if the thought inside us is that “God exists, but has flaws”? Well, you see where this is going.

There is a cheeky way out of this conundrum: Simply accept Descartes’ definition of God. This is a reasonable way to understand philosophy.

To prove the existence of something, one must begin with a definition of that thing. And if we are to follow Descartes’ reasoning, then we should accept his definition too.

Omissions from this review

I have omitted some of the things from this essay that I found to be very unconvincing.

God Exists, Approach 2

In Chapter 5, Descartes considers the question about whether God exists once again and “proves” this using another approach. But the second approach contradicts some assumptions I made to understand the original approach. I can’t bring myself to reject the assumptions I made for the first proof and engage with the second proof, which asks me to assume the opposite thing. A thing and the opposite of that thing can’t both be true at the same time, and hence, only the proof that involves one of those things can be accepted.

Briefly, the author says that God exists because I understand him as clearly and distinctly as I understand that a Triangle exists.

I have 2 objections to this line.

Firstly, to justify Descartes’ original line of reasoning, I started with the idea that I don’t understand anything and don’t accept any thing’s existence unless proven. So, in that frame of mind, I can’t really start from the idea that a Triangle exists.

Secondly, I don’t think that I understand God “clearly and distinctly.” In fact, it is most unclear to me what God would look like or is capable of, unlike a Triangle which I understand much better: I know that it has three sides, I know that the sum of two sides is more than the third, I know that the sum of all angles will be 180 degrees, and so on.

So, this approach was even less appealing to me than the first one.

Errors in Judgment

This was another place where there were too many circular reasoning steps to justify including it in the review. He starts with the question,

Why do I commit errors in judgment, when I accept that God, who is perfect and infinite, created me?

To this, he has a simple solution: I am incomplete and imperfect. So, I simply can’t understand the answer to my question even if it were presented to me. So, I should just not bother.

After giving this answer, he goes on to provide some suggestions for how we might avoid errors in judgment and these are positively insane:

Refrain from making a judgment about anything that is not clearly and precisely understood by the intellect
Withhold judgment on anything whose truth is unknown

That is no way to live!

Proving that God is Perfect and Infinite

Descartes makes (what I can only decode as) a halfhearted attempt to prove that God is perfect and infinite. He says that while it is possible for me, a finite being, to be created by someone who is only marginally more finite than me, it is not the truth.

Because there are so many things on Earth and there is the Earth itself and so on. So, everyone and everything’s cause of existence is necessarily infinite.

If you are not convinced by that argument, neither was I.

Another thing he says to prove it is, “the idea of God [as a perfect and infinite being] is innate within me.” Clearly, this statement is even less useful than the previous one. A child that hasn’t been initiated into the ways of religion and common thoughts about God does not have this idea innate within them; it is society and the child’s observation of society that gives rise to this idea inside her.

Dreams vs. Reality, resolved

In Chapter 6 of the book, Descartes tries to resolve the split between dreaming and reality. However, I did not understand this meditation at all. He pieces together several arguments from his previous meditations and proves it somehow.

Quotes

What about thinking? Here I make my discovery: thought exists; it alone cannot be separated from me. I am; I exist—this is certain. But for how long? For as long as I am thinking; for perhaps it could also come to pass that if I were to cease all thinking I would then utterly cease to exist. At this time I admit nothing that is not necessarily true. I am therefore precisely nothing but a thinking thing; that is, a mind, or intellect, or understanding, or reason—words of whose meanings I was previously ignorant. Yet I am a true thing and am truly existing; but what kind of thing? I have said it already: a thinking thing.

But what then am I? A thing that thinks. What is that? A thing that doubts, understands, affirms, denies, wills, refuses, and that also imagines and senses.

Finally, as to my parents, even if everything that I ever believed about them were true, still it is certainly not they who preserve me; nor is it they who in any way brought me into being, insofar as I am a thinking thing. Rather, they merely placed certain dispositions in the matter which I judged to contain me, that is, a mind, which now is the only thing I take myself to be. And thus there can be no difficulty here concerning my parents. Indeed I have no choice but to conclude that the mere fact of my existing and of there being in me an idea of a most perfect being, that is, God, demonstrates most evidently that God too exists.

I perceive that the power of willing, which I got from God, is not, taken by itself, the cause of my errors, for it is most ample as well as perfect in its kind. Nor is my power of understanding the cause of my errors. For since I got my power of understanding from God, whatever I understand I doubtless understand rightly, and it is impossible for me to be deceived in this. What then is the source of my errors? They are owing simply to the fact that, since the will extends further than the intellect, I do not contain the will within the same boundaries; rather, I also extend it to things I do not understand. Because the will is indifferent in regard to such matters, it easily turns away from the true and the good; and in this way I am deceived and I sin

However, as far as God is concerned, if I were not overwhelmed by prejudices and if the images of sensible things were not besieging my thought from all directions, I would certainly acknowledge nothing sooner or more easily than him. For what, in and of itself, is more manifest than that a supreme being exists, that is, that God, to whose essence alone existence belongs, exists?

The full quote is “I doubt, therefore I think; I think, therefore I am.” ↩
Descartes counsels the reader early in the essay that they should read ahead only if they are willing to put their prejudices aside and meditate with him on the subject at hand. He also says “Discerning men are few and far between,” which was probably the only light, humorous line in all of the 77 pages. ↩
I am using “axiom” here in its dictionary sense. Namely, “a statement accepted as true as the basis for argument or inference.” ↩ ↩²

India's National Monetization Pipeline

2021-12-19T00:00:00+00:00

The National Monetisation Pipeline was a policy announced by the BJP government in India in August 2021. This policy was not widely covered in any of the national newspapers, and there were no policy analyses that went deeper than the headline numbers highlighted by the government. While some articles started coming out in September with more details about the government’s expectations, I couldn’t find any criticism of the government’s approach. A tired comparison to Australia’s asset monetisation plan was everywhere, but this did nothing but scratch the surface. (The same comparison was repeated in multiple publications including The Print and India Today.) This post is a summary of my understanding of the policy after reading some detailed coverage in the September 27, 2021 issue of the India Today magazine.

This post is broken up into three sections: What, Why and How. I have included footnotes with sources for all the figures that are mentioned in this post.

What?

India’s National Monetization Pipeline (NMP) was unveiled by the government in the last week of August 2021. It is a policy that aims to raise Rs. 6 lakh crores¹ over the 4 years between 2021 and 2025.

The pipeline plans to put up existing, operational government-owned assets for monetisation. Private companies can get “revenue rights” to these assets, which will enable them to enhance them (such as widening operational roads) and provide new services using them (such as new trains on existing railway tracks). The ownership of the assets will remain with the government. And when the tenure of monetisation ends, revenue rights will revert to the government. In effect, the government is leasing out its assets to private players.

The question for private players who are considering any investment is clear:

Can they use the asset to generate enough revenue to recoup the capital that they invest?
If they can recoup the capital, can they make a profit on the asset in the limited time for which they will hold the revenue rights?

Government’s expectations

For assets that are monetised, the government expects the following three things from the private sector entity:

Upfront capital
A share of revenue generated from the asset during the period of monetisation
Investment commitment towards the monetised asset

The government expects the private player to increase efficiency, enhance the quality of service, and increase userbase through service improvements, so that they can charge enough to make money despite having to pay a share of their revenue back to the government. In reality, most private players that will monetise an asset will probably use all three strategies.

Connection to National Infrastructure Pipeline (NIP)

NIP is the flagship policy that the Indian government unveiled on August 15, 2019 (India’s Independency Day). This policy’s goal was to invest Rs. 100 lakh crores into infrastructure projects in India during the period from 2019 to 2025. This policy is planned to end at the same time as the NMP.

NMP is one of the ways in which the government is raising money to invest in the NIP. (The government’s revenues from NMP will be re-invested in “new infrastructure” that will be built as part of NIP.)

The NIP’s headline number was ambitious. The total infrastructure investment in India in the preceding 5 year period from 2013 to 2018 was Rs. 46.48 lakh crores, according to the Department of Economic Affairs (DEA).² In effect, Modi’s plan was to double infrastructure investment.

Surprisingly, I could not find any summary of how this policy is going after 2019. The official policy document marks several of the important Key Performance Indicators as “TBD (To Be Decided).” The latest mention of this policy was in the Budget speech in February 2021 where new projects were added to NIP. It is already 2 years since the policy’s implementation period began, but there seems to be no mention of it anywhere on Government websites or any media coverage of how much investment has come in through this policy and whether the government is meeting its phasing goals for the project.

My read of the numbers is that the government is falling short of the capital expenditure planned under the NIP:

Period	Planned CapEx from Centre under NIP³	CapEx as in Budget 2021-22⁴	Percentage of CapEx achieved
2019-20	5.62 lakh cr.	3.35 lakh cr.⁵	59.60%
2020-21	8.39 lakh cr.	4.39 lakh cr.⁶	52.32%
2021-22	8.31 lakh cr.	5.54 lakh cr.⁷	66.66%

Comparing Expected Revenue

To put this number in context, here are some headline figures for the Indian economy. All figures in the second column are in lakh crore INR (= 1 trillion INR). The figures for government revenue and expenditure are from the “Actual” values of 2019-20, the last year which has published “Actuals” figures and was not affected by the Covid19 pandemic. Figures are taken from the PRS India Union Budget analysis.

Header	Value (lakh crores INR)
NMP’s expected revenue (5 years)⁸	6
NMP’s peak yearly revenue (2023-24)⁹	1.8
Actuals, 2019-20
Government receipts (without borrowing)	17.5
Government borrowing	9.33
Government expenditure	26.86
Defense expenditure	4.52
Education	0.89
Railways	0.69
Government subsidies (petrol, food, fertiliser, others)	2.62
India’s 2019-20 GDP	145

So, the government’s peak yearly revenue from NMP would cover the government’s expenditure on Education and Railways. Looking at these figures, it’s clear that the NMP is not as ambitious as the NIP.

Why?

There seem to be two major reasons for this policy:

Public sector management of public assets is inefficient: This is tacitly accepted throughout the NMP policy document, especially in sections where the policy talks about “unlocking value” in existing assets. The government recognizes that efficient management of public assets is essential to the economy’s growth. However, to achieve this, the management is outsourced to private players.
Government borrowing is at a limit: For the NIP, the government must raise Rs. 100 lakh crores in the 2020-2025 period. Until now, it seems that the main method for raising this capital has been government borrowing. (I could not find concrete figures for how much of the borrowing has gone into NIP projects.) The debt-to-GDP ratio is just above 62.9% in 2020-21. Moreover, over the remaining period of the NIP, until 2025, the government plans to bring this ratio down to 60%, according to the recommendations from the 15th Finance Commission. This means that the government has to pay back about Rs. 4.5 lakh cr.¹⁰ of the oustanding liabilities to achieve their debt-to-GDP ratio goal. So, more borrowing is not the solution to fill the gap in NIP capital. The government sees monetizing existing assets as the only viable option.

One thing jumped out at me after reading through the literature about both NMP and NIP. NMP’s underlying assumption is that the private sector is better at efficiently maintaining infrastructure compared to the public sector. So, one is forced to question the government’s wisdom in trying to build more public infrastructure under the NIP, which will be managed by the public sector, while it is accepting that public sector management is inefficient in the long-run.

How?

Details about the implementation of this policy are completely missing from the original policy document. Most media coverage about the NMP simply skips covering this part, because there is nothing to be said.

Firstly, the details that are missing are: what kind of rights will be part of “revenue rights,” how much upfront capital and revenue share the government will demand for each type of asset. One has to assume that these numbers will never be set in stone, because the private sector will expect them to be negotiable based on their existing scale. They will also hope to get a “wholesale” discount, if they decide to invest big in NMP. The government would not want to tie its hands during these negotiations by publicly announcing a minimum value for any of these figures. So, the public will probably find out about NMP projects only after the negotiations have been completed and the government has already made a decision about these numbers.

Secondly, whether the private sector is even interested in taking over management of mismanaged public assets, investing capital in them and trying to turn them around, is up for debate. There has already been some criticism of the two major asset classes that are up for monetisation:

Roads: Roads make up 27% of the expected revenue from NMP. 26,700 kms of roads that are operational or under construction with at least 4 lanes are up for monetisation. These roads are part of the National Highway network and the toll collection rights have been retained by the National Highway Authority of India. The criticism of this asset class is that 4-lane roads are already being managed efficiently by the NHAI, and that as long as toll collection rights remain with the NHAI, it seems unlikely that private players would want to put in any unrecoverable capital in the enhancement of this asset class. (As in the long term, NHAI will reap the benefits of the private player’s capital investment.)
Railways: Railways make up a further 26% of the expected revenue from NMP. 400 railway stations and 90 passenger train operations are up for monetisation. Currently, Konkan Railway seems to be the only railway entity in the country which has atleast one private stakeholder involved in its management. Given this relative lack of experience in the private sector, getting them involved in operations through asset monetisation seems akin to throwing them into the deep-end of the swimming pool. An approach in which entities that have experience managing operations can work with entities that don’t have any experience in an effort to build the private sector’s experience would be preferable. This approach would also give them a close look at the potential for profit in Railways operations.

Policies die in the implementation phase, where ambitious projects transform into reams of paperwork making their way through an incompetent, circuitous and intolerably slow bureaucracy. Private sector players are unable or unwilling to invest the resources required to cut through this regulatory red tape. The very existence of the red tape will tilt the field towards a very small number of players, and this policy might result in an oligarchy. This concern has been raised before, and the government seems to be willing to accept the risk of oligarchy as long as they are able to get the capital required to enact wider benefits. In a nod towards the influence of the reigning monopolists in India (namely, Ambani and Adani), the leader of the opposition did not mention their name in a press conference in which he was criticizing the NMP. Instead, he chose to refer to them indirectly as “2 or 3 people that you already know about.”

With a goal of raising Rs. 0.88 lakh crore⁹ by April 2022 under the NMP, missing implementation details raise questions about the government’s ability to achieve their target.

1 lakh crore = 1,000,000,000,000 = 1 trillion. Rs. 6 trillion = USD 78 billion ↩
Author’s calculations using Figure 7 on pg. 26 of NIP report from the Department of Economic Affairs. ↩
Table 3 in DEA report. This is 39% of the planned capital expenditure, because NIP’s investment is shared between the centre (39%), state (40%) and private sector (21%). ↩
“Budget estimates of 2021-22 as compared to actuals for 2019-20” section in PRS India’s Analysis of Union Budget 2021-22. This document was tabled on February 1, 2022. ↩
Figures from the Actual estimates for 2019-20 ↩
Figures from the Revised estimates for 2020-21 ↩
Figures from the Budgeted estimates for 2021-22 ↩
Figure 5 in the NMP policy document. ↩
Figure 7 in the NMP policy document. ↩ ↩²
2.9% * Rs. 145 lakh cr = ~Rs. 4.5 Lakh cr. ↩

Monthly Recommendations (November 2021)

2021-12-11T00:00:00+00:00

The theme for November’s list is Nature. Movies starting with Jurassic Park have tried to show humans that Nature’s powers are wielded at the most uncomfortable times for human beings and that there is really no consideration for us. This does not stop us from wanting to change the natural course. And this forceful change in nature’s course that we effect through our actions, our scientific understanding of fundamentals, our engineering acumen to build huge structures, and our knack for planning long, complex adventures with far-fetched goals in mind is the theme of this month’s recommendation list. It includes one article about a ship that got stuck in the Suez Canal, a team of people who want to set a world-record and thus circle the globe 1.5 times in their pursuit, and the story of a government’s need to do data-driven policy gone haywire.

Six Days in Suez: The Inside Story of the Ship That Broke Global Trade (Chellel, Campbel, Ha)

Bloomberg Businessweek (archived). 24 Jun, 2021

Great investigation and very interesting notes from the people who were deeply involved in the operation to dislodge the ship.

The Ever Given disaster was perhaps one of the most interesting news story of the past year. It was a welcome relief from the constant grief that COVID19 stories cast on all news coverage. It was also a problem that seemed so tractable. After a year of coverage about variations in virus genomes and vaccine efficacy, the sudden blockage of a canal because a ship got lodged in it and blocked it seemed almost unreal. At the story’s beginning, there were very few images and most of the coverage repeated the same few refrains. Bloomberg managed to get the first eyes on the ground and returned some footage that showed the massive scale of the problem. Soon after, the outlook was pessimistic enough that shipping major Maersk decided to re-route ships through the Cape of Good Hope. The solution to the sideways ship was also surprisingly quick. Helped along by a high tide caused by a full moon, the ship was dislodged in about a week. The backlog was cleared in another week. The rerouted ship would take 3 weeks to travel the distance that is covered in about 14 hours through the Suez Canal. This article is an amazing recap of the whole incident.

It’s not perfect, but that’s the story with all media coverage. The authors don’t meet the mark with their knowledge of the major languages in various regions. In particular, a cursory search would have told them that a huge number of children in India study in schools where English is the medium of education and thus, despite not being my native language, I continue to write blog posts in English. A note about the background of the people on the bridge would have been useful, such as the regions that they studied in or the language that they completed their naval training in. (I am very confident that these seamen must have completed their training from institutes in India and given their exams in English.)

After climbing aboard, the two Egyptian pilots were led up to the bridge to meet the captain, officers, and helmsmen, all of them Indian, like the rest of the crew. According to documents filed weeks later in an Egyptian court, there was a dispute at some point about whether the ship should enter the canal at all, given the bad weather—a debate that may have been hampered by the fact that English was neither side’s first language.

Also, I don’t see why language is an issue here. These are men trained to make snap decisions in crisis situations such as these. Their language will not hamper their communication as there is a set hierarchy that must be followed anyway.
Five Oceans, Five Deeps (Taub)

New Yorker (archived). 18th May, 2020

The incredible non-fiction piece about a team of researchers and world-record setters who traveled all 5 of the deepest points on the planet’s oceans and descended to the bottom in their vessels. This is a story of survival, the trials that come along the way, and the importance of world records. In all the ways that one can get their name etched into history, world records are perhaps the most accessible to the normal person. Of course, it’s all relative. This particular mission required a lot of resources, and the world record was set only because the people involved in the project could come up with that money. This article also touches on that point. The other ways of getting into the halls of history involve politics or sports, both of which require atleast a little bit of skill that the person must be born with. There are some who like to believe that politics is completely random because people form their opinions well before they vote in an election and that sports is completely random because athletes can be good if they are trained early and relentlessly. I am not in both those camps yet and firmly believe that something innate is a requirement; however small that might be, there’s something that’s required. To someone who believes that, setting “one of the last meaningful records” would definitely seem to be worth the hefty price.
Black, No Sugar. (Gopnik)

New Yorker (archived). 27th April, 2020

Coffee is everywhere and it’s the most common drink that one finds in the newest of places. There is a lot of fuss over the taste of coffee; I don’t have a taste palette refined enough to understand even 10% of the conversations about coffee among connoisseurs. But the history of coffee is a fascinating topic to me because it tracks the emergence and spread of a caffeine delivery system. Gopnik chronicles the books about the corrective history of coffee that questions coffee drinkers around the world about the moral costs of their constant consumption, and also questions the basis of their consumption itself; the relation between caffeine and productivity may have been slightly overstated by the coffee lobby inside political institutions, making it a phenomenally easy commodity to trade. Gopnik uncovers such tricky truths in his piece.
The Pandemic Made Cities Quieter, But Not Less Stressful (Sullivan)

Bloomberg Businessweek (archived). 2nd September, 2021

Sound is everywhere but we barely notice it in most situations. When we do notice it, it is with either the at most pleasure or the vilest displeasure. The Vox, Explained episode about Music is also a great reminder of this core role that sound (noise and music) plays in our lives. With the reduction of some city noises over the past year, as economies came to a screeching halt, other noises have become prominent. The uncomfortable silence in a city is comparable to the uncomfortable silence inside your office, where you can here your own voice clearly, for there is no other background noise. This is an article about how soundscapes and well-designed cities might be the way to solving the noisy city problem, once-and-for-all. If we would want to make the city less noisy is another question altogether. I am not yet convinced that we need to live in a relatively silent city to be less stressed. To me, the noise that runs through cities is the key to separate them from the countryside.
India’s Air Pollution Mitigation Strategy Needs More Direction (Shah)

The Wire (archived). 7th June, 2021

Government policies take a lot of time to be drafted and even when they become bills they are huge documents without a straightforward explanation of their goals. This is clearly hard; I concede that it might even be impossible: How do you design a straightforward system for hundreds of millions of people where even a corner case that occurs 0.1% of the time will affect the lives of 100,000 people?

However, the dense regulatory framework benefits corporations that have the revenue to pay lawyers to parse through policies and find ways to use them to their advantage. This article views the problem of policy making from the government’s side: Why does the government not make a policy to reduce pollution in India’s metro areas? The government is criticized for taking steps which are not backed by “real numbers.” To get these real numbers, the government commissions studies and forms committees.

This article rails against such actions; and specifically, again “source apportionment studies,” which (presumably) attribute the existing pollution to various sources and try to apportion blame to each source. The results of such studies are used as the basis of “data-driven policy.” However, as the author demonstrates, these studies have not discovered anything useful or new in the 15 years between 2001 and 2017. Institutions like IIT Madras and IIT Kanpur are involved in the conduction of these studies; I know that studies like these are a major source of revenue and research papers for the academics at these institutions. Publishing at a high rate is one of the assured paths to great publicity for academics; and thus, the incentive structures line up perfectly between governments and academics. These studies deliver no value to the government. So, the incentive structure to reduce air pollution? Well, it is dysfunctional.

Monthly Recommendations (October 2021)

2021-11-02T00:00:00+00:00

I have tried to stay away from COVID-19 on this blog. Most people are writing nearly constantly about it and I don’t have many original takes on this topic. But this month, I reverted back to a way-of-life that was typical for me before the pandemic began. I traveled to another city to meet relatives; I went on a vacation; I traveled to some places in the city that I am in; I went out to restaurants and dined-in. These experiences signal a return to normalcy which seemed unlikely in March 2020, 18 months ago. These experiences also shined light on a change within myself: I have gotten used to not being among strangers. Going back to a previous lifestyle, seeing and listening to strangers again has been an interesting return to normal. (I am glad that the “new normal” was transient.) So, this month’s theme is a receding pandemic. I look back at some of the articles which I read at various milestones during the pandemic, starting from the beginning of the pandemic in Japan last year, through the emergence of the vaccines, and right up to the most recent article about the long-term effects of this crisis.

Existential Inconvenience (Dyer)

New Yorker (archived), 23 March, 2020

This article was written right when the pandemic was beginning. Realizations like “knowing that you will not be able to do something is better than knowing that it is still possible but unlikely” seem obvious. But having them articulated at you is also clarifying. I am not sure why that is the case. (I am trying to figure that one out still.) The author calls these “existential inconveniences,” I think that’s an apt name. They don’t really rise up to the level of problems (For e.g., I could not see Bob Dylan perform live in Japan in April 2020 because his show got canceled.) but they are at least inconveniences.
Memoranda (Rozzo)

New Yorker (archived), 18th May, 2020

War analogies are morose. But they are also effective because every war has an enemy, and all wars end, even the ones that seem to drag on forever. These two crutches are required for prolonged belief in the proverbial “light at the end of the tunnel.” This article is a set of poems written by a soldier who arrived in New York during the worst art of the pandemic there. The soldier (who remains anonymous) was able to write poetry to capture the moment and escape from it himself. The poetry is simple and beautiful; at times, it simply restates facts. But I am not sure I would be able to write something this good when I need to focus on crisis management; and this is the main reason I am recommending this piece. It’s a reminder about what’s possible when one is in an unprecedented situation.
Weaponizing Uncertainty in COVID-19 Lab Leak Theory (Rogers)

Wired Magazine (archived), 20th May, 2021

There was a recent report from the US intelligence community about the origins of COVID-19. (I have not read the full report; I have read the first few pages and reporting about it.) Indulge me and pretend that new evidence matters. Indulge me further and pretend that new evidence has the power to change already formed beliefs/opinions/facts. The report can be used to prove any opinion that the reader has already arrived at. For e.g., for those who believe that China is not to blame for the disastrous handling of the pandemic in US, Europe, and UK, they can go to this line:

Area of Broad Agreement: Virus not developed as biological weapon; Virus not genetically engineered

And for those who want to blame China, they can use this line:

China’s cooperation most likely would be needed to reach a conclusive assessment of the origins of COVID-19. Beijing, however, continues to hinder the global investigation, resist sharing information, and blame other countries, including the United States.

Even though the report goes on to say that “Beijing’s lack of cooperation on origins not diagnostic of either hypothesis,” this second point is mostly glossed over in second-hand reporting of this report. It says that the report does not explicitly blames China for the virus, but it does blame China for not sharing enough information. The retort that everyone is familiar with rises: “If they don’t have anything to hide, why are they not sharing all the information that they have?”

I will not pretend to have any answer. I am fairly certain that there is no answer. Even if there is one, this answer will not be found. So, what is the point of all this coverage of the virus’ origins? Biden needs to win the mid-terms; Europe needs a scapegoat for their disastrous handling of the pandemic; Most other countries have upcoming democratic elections where voters will demand to know why their government failed where China succeeded. This take is clearly cynical; and this article supports this take from the point of view of weaponized uncertainty: Not being able to find something casts suspicion on the institution which most likely has the coveted evidence.
Moderna’s Next Act Is Using mRNA vs. Flu, Zika, HIV, and Cancer (Langreth)

Bloomberg Businessweek (archived), 14th July, 2021

How did we get a vaccine for the coronavirus so soon? The strategy of using a vaccine as the path out of COVID-19 was considered reckless and foolhardy in March 2020. “Vaccines take years to make. The earliest period of time in which we have been able to develop safe vaccines in the past was 4 years.” Policy makers had different reactions to these warnings from vaccine scientists. Vaccine manufacturers like Pfizer and Moderna stayed quiet and worked on finding the vaccine. The announcements in November 2020 about the 90+% efficacy that vaccines had shown in Phase 3 clinical trials was perhaps the best piece of news in the past year. This article explains why the mRNA approach to vaccines is modular and configurable, features which are at the core of scalability in software, and apparently also in vaccine development. This article elaborates on how mRNA could turn into the ticket out of future pandemics and existing incurable diseases.
We’re Losing Our Humanity, and the Pandemic Is To Blame (Smith)

ProPublica (archived), 7th October, 2021

The pandemic has been the comfortable scapegoat for a lot of uncomfortable truths that have come to light in the past few months. Particularly, about “losing humanity,” I wonder if the changes we have seen in behaviors over the past 1.5 years were just latent tendencies which were triggered by the pandemic. The ongoing pandemic is a mix of three things: a once-in-a-lifetime crisis, a conspiracy theory breeding ground, and an event that increases fervor about already held beliefs. The period of lockdowns in 2020 was also the first time in my lifetime where I had limited contact with strangers for that long a period of time. This has made returning to the world and interacting with strangers again a shocking experience. Recently, returning from a vacation, I found that many of the strangers around me were in an anxious and nervous state which made them take actions which lacked empathy and considerateness. This was not something that I was noticing for the first time; nevertheless it hit me like a rock wall and left me confused for a few days. For that feeling of confusion, I am certain that the pandemic is to blame.

Climate Change Fever, Late 2021 Edition

2021-10-30T00:00:00+00:00

If you haven’t caught the climate change fever yet, you almost certainly will catch it in the next few weeks. Even a cursory glance of the news will familiarize you with COP26. As the climate conference approaches, Western publications have gone all-in and bet that the conference will generate a large amount of news. They are right about this, the conference is definitely going to be a hotbed of some stunning news stories. Some countries might announce new targets, others might go back on their existing targets or not commit to anything new, etc. Every change in policy will be followed extremely closely and dissected and analyzed from every possible angle. Only one question will remain unanswered: What is the point of all this hoopla?

Climate change science and climate change activism are no longer distinct. Scientists and activists used to occupy different positions in the discourse, as Matt Yglesias has pointed out before. Science is about trade-offs, because very few things come at 0 or near-0 costs. Activism is about convincing other people to share your beliefs and implement your proposals, and thus, selecting evidence that will help you achieve that end and ignoring evidence that will hurt your ability to achieve your goals is ethically acceptable to an activist. Returning to a Matt Yglesias quote,¹ “If you are an activist and you are admitting that there are trade-offs to your proposals, then you are doing activism wrong.”

When it comes to the topic of climate change, scientists often speak like activists, issuing dire warnings about the devastating effects of a warming planet. Activists speak like scientists, armed with information that has become ridiculously easy to access and easier still to understand, as it has been boiled down to a few numbers. Never mind if the numbers actually tell the story that a naive interpretation of them would suggest.

Politicians occupy a sticky position in this discourse. Their job is to reflect the majority’s thoughts back at their voterbase. On an issue such as this one, it is nearly impossible to poll the right number of people and find out what the majority thinks. This has crippled the political response.

Some activists have become politicians but have not changed their role as their constituencies are so “in the bag” that what they say or do is completely immaterial. AOC’s recent questioning of executives from big oil was a good instance of toxic, futile posturing. I believe that smart, sarcastic coverage of Big Oil has always hit the mark in convincing people of the shady roles that Big Oil and their lobbyists play in the lives of the citizenry and how they are warping (education, energy and economy) policy. Laura Grey’s segment from the show, The Opposition with Jordan Klepper, is a good example of such coverage.

In the face of opposition even from members of their own party in countries like America, politicians have ended up facing the reality that is clear to neutral observers: The economy is deeply tied to the technologies which the climate change activists say must be eradicated and the economy is one thing that has real impact on people’s lives and swings elections.

The media has undoubtedly taken a side. Articles about how COP26 is the place where most problems will be solved are commonplace. “The importance of this summit cannot be overstated,” “It is high time that the leaders of the world get together and do something about this existential threat.,” “Rich nations have a responsibility to the global poor,” “Poor nations that continue to pollute must be penalized by rich nations,” and so on. The unending contradictions are impossible to parse.

But there is one specific talking point that I want to delve into.

Who Is The Biggest Polluter?

Most publications and activists point at India and China as the biggest polluters. This is an intentional jibe and a mischaracterization of the issue at hand.

Here’s the table of emissions and populations, sorted by annual emissions. (A note about abbreviations can be found in the footnotes².)

Country	Annual GHG emissions (billion tonnes)	Population (billions)	GHG emissions per capita (tonnes)
China	10.17	1.402	9.22
America	5.28	0.329	20.20
India	2.62	1.380	2.38
Japan	1.11	0.125	11.59
England	0.369	0.067	9.20

Indeed, China appears to be the largest polluter. Sorting this table by per-capita emissions tells a different story.

Country	Annual GHG emissions (billion tonnes)	Population	GHG emissions per capita (tonnes)
America	5.28	0.329	20.20
Japan	1.11	0.125	11.59
China	10.17	1.402	9.22
England	0.369	0.067	9.20
India	2.62	1.380	2.38

Both America and Japan are far bigger polluters, per resident of their nations. Looking at per-capita emissions seems like the obvious method for assigning blame because the number of people in a country is the only constant which can’t be changed, in the lack of drastic changes like a war.

The high GHG emissions are due to the level of advancement of these countries, which entitles them to uninterrupted electricity, cheap gasoline, higher levels of disposable income leading to far more air travel, etc. These advancements are objectively good things for the populations of these countries and not something to be looked at grudgingly or that countries should be penalized for. So, hiding behind the “Annual GHG emissions” number is a dodge from the media. They recognize the failure of the US and Japan to take any action; and they channel this recognition into an effort to reassign blame to China, a poorly understood nation without the power of a persuasive English media ecosystem.

As the failure of the degrowth movement (even in the most liberal parts of the world) has shown, the solution that holds sway among populations is not one which involves a compromises of any kind. So, any rhetoric about how “human beings have hurt the planet and its time we suffer to redeem ourselves” is useless.

Griffith’s vision is a solution that involves retaining the same level of comfort but changing the energy sources to ones that have lower emissions for producing the same amount of energy. But as Griffith admits, the problem is not one of engineering, it is one of political will.

So, as you look at the coverage of the COP26 summit, bear in mind that the summit exists only to convince people in the speakers’ countries that their country is doing enough and that the cause of inaction is abroad. The likelihood of any major treaty emerging from this summit is low, especially given that the world’s largest polluters are stuck in political limbo with a polarized voter base.

There will be a lot of “symbolic” announcements³ and these will be celebrated in lieu of the real action that this conference was supposed to coerce out of nations. The blame-game against China and India’s refusal to commit to any new net-zero goals will continue unabated.

This quote is from Matt Yglesias’ appearance on the Ezra Klein Show podcast in late 2020. ↩
Note: 1. GHG = Greenhouse Gases; 2. CO₂ equivalent: Equivalent emissions of CO₂ gas if the emitted gas is not CO₂ ↩
Such as Saudi’s commitment to Net-Zero in 2060; a commitment that does not involve a reduction in crude oil production and is focused only on the consumption of fossil fuels inside Saudi Arabia. ↩

America: An Unreliable Ally

2021-10-24T00:00:00+00:00

The American political system is strange. It has three branches; the Executive, Congress and the Judiciary. Judges within the judiciary are appointed by the Executive with approval from Congress. The President is the highest official in the Executive branch and elected directly by the people. And all members of Congress are also directly elected. All the lawmaking power lies with Congress and it is generally very hard to repeal laws which have already been enacted. On the other hand, the Executive branch is a fickle-minded body where the current President’s whims and fancies are the primary deciding force. Treaties are signed by the Executive branch. And wars begin and end at the Executive branch’s will. This makes America an unreliable republic to have any kind of long-term agreement with.

Democracies crave stability. Economies are the most important things that governments manage, and long-term economic policy can be drafted only when one’s allies are clearly known and their behavior can be reliably anticipated or they can be trusted to give sufficient warning of major policy changes. There are more problems that governments generally want to solve like providing the right amount of nutrition to citizens and enabling access to modern healthcare, emanating from Research and Development efforts in allied countries. All these things hinge upon a reliable ally.

The US-UK relationship is special¹. This in turn makes sudden policy changes such as Brexit in UK difficult to navigate for the allied country. Similarly, the abrupt American exit from Afghanistan left Britain in a strange spot. They discovered that refugees from the war-torn country were going to come to various countries in Europe and might soon be eligible to travel to and work on British soil. This puts the policies that both countries have for their own countries in jeopardy, as these sudden excess/deficient flows of goods and people will drive a wedge through established policy.

The Executive branch signs treaties which the Executive branch is free to leave. The reversal is not at all stunning, and in fact, is the obvious solution to an ideological disagreement. If your predecessor entered a deal that you believe is not a good one, just leave. Being in the powerful position of being the largest economy and a leader among English-speaking nations (which covers most of the world, due to British colonial history), the US seldom has to worry about actual impacts inside America.

I stress the word actual here because the Media coverage will insist on the “stunning policy reversals,” even though a self-interested actor (such as a politician) will think of this reversal as the obvious step. Journalists seem to have a soft-spot for symbolic moves. I am incensed by this false emphasis on “symbols” within policy, where the final policy that gets enacted is what matters, and everything else is just fluff. (Sidenote: The US government is good at creating fluff: Biden managed to improve his approval ratings abroad within just 3 months of taking office. This approval rating abroad started falling later in his presidency though. And the global media is exceptional at consuming fluff, both of the self-aggrandizing sort and otherwise.)

Obama entered the Iran Nuclear Deal and the Paris Climate Accords in 2015. Both of these were touted to be a big deal² at the time and for several years after the fact. Trump took office and swiftly moved to get out of both deals. He gave various reasons for his departure. From the point of view of other parties to this treaty, the reasons are immaterial and no amount of ridiculing Trump³ or the Republican Party is of any real use. The consequential decision had already been made.

Finally, going further in from foreign policy to America’s domestic stance on issues, Biden’s famous “executive order” purge on his first day in office was another symbolic move that served to confirm America’s unreliable status as a “partner” in anything. I was surprised that the Biden administration made a big deal about directly reversing 9 Trump executive orders; this kind of stunning policy reversal is the last thing other partners want to see.

With a falling approval rating at home and abroad and an inability to build a consensus for domestic policy, the Biden administration has failed at their “America is Back” campaign. Facing a grueling mid-term election in just 1 year, in which they are projected to lose control of Congress, it is questionable that any significant policy will be enacted in Biden’s remaining presidential term.

However, this is in no way a hindrance for people that want to see America do something big: the Judiciary branch’s top body, the Supreme Court, has just begun a term with some very big cases scheduled. And Congress is probing both Facebook and Google for (what I approximate to) shady monopolistic business-running behavior.

“The United States has no closer ally than the United Kingdom, and British foreign policy emphasizes close coordination with the United States.” ↩
I can not comprehend why a climate agreement where countries set non-binding targets which they can choose not to adhere to is a big deal. But hey, your local journalist says its a big deal and so it is. ↩
Colbert’s coverage of Trump was vitriolic and unwatchable during most of this period. ↩

Monthly Recommendations (September 2021)

2021-09-30T00:00:00+00:00

This month’s theme is the Chinese state. Lately, there has been a lot of news out of China. As Xi Jinping guides the China story towards the inevitable conclusion of overtaking the US economy in size, he has started announcing reforms that look unthinkable to many but deal with the key underlying issues in late-stage capitalism. Rising tuition fees and the unsustainable costs of tutoring are a key issue in India. Coaching classes that train students for the entrance exams of leading engineering and medical colleges cost about INR 50,000 per annum, which is a third of the annual per-capita income of India. China’s crackdown to make these coaching classes non-profits is the most refreshing reform that I have seen imposed on this industry in the past decade. Another industry that has been on the rise in India is online gaming and online gambling. The rise of this industry has made strange cases unusually frequent. (These cases remain the exception and not the norm.) China’s recent regulation on gaming time for minors is an ambitious step. I am not convinced about its utility; but I can see that drastic changes are required to curb Big Tech’s unfettered access to data collection and the Chinese state does not balk under pressure from Big Tech lobbyists. This alone is encouraging to me and I hope that it convinces democratic governments to try something in their own jurisdiction.

Life on Lockdown in China (Hessler)

New Yorker (archived), 30 March, 2020

China’s system works in times of crises. Societal responsibility, a collectivist mindset and swift government action was the answer to COVID19. Every country that tried other approaches failed at getting a good outcome for society. This article also shows the early opposition to school closures. (Even as early as March 2020.) It also ends on a hopeful note during a short period of relaxation in lockdown regulations.
China Crushed Jack Ma, His Fintech Rivals Might Be Next (Chen et al)

Bloomberg (archived), 25 June, 2021

The action that the Chinese government took against Jack Ma (his mysterious disappearance, his reappearance, his changed positions) right after he voiced his opinion about the Chinese political system was a stunning reminder of the depth of state power. Just as this article prophesized, the Chinese government has gone ahead with a complete ban on cryptocurrencies (ostensibly a “Financial technology” business).
China’s Risky Bet on Afghanistan (Marlow, Curran)

Bloomberg (archived), 24 August, 2021

China is flexing its muscle on the global stage and warzones seem like the best place to make a big profit, at a modest investment, owing to the high risk nature of the investment. This is due to the chaotic nature of the post-war period in general. In the case of Afghanistan, the chaos is compounded by the debacle that was the American withdrawal from its 20-year war in August 2021. Biden and his administration have tried hard to reinforce the idea that 120,000 people were evacuated from Afghanistan during the withdrawal. They have repeatedly distanced themselves from the original commitment to end the war and talked about how they were simply abiding by an agreement that their predecessor had drawn up. They have also taken pains to emphasize their commitment to democracy and women’s rights (even as questions have risen about their domestic commitment to both these issues.)

It is clear that America lost the war, left the country in shambles and handed the country over to a government whose goals are unclear and whose conduct is erratic.

Strangely, Biden’s remarks about ending the war did not reflect the celebrated American exceptionalism that appeared to me to be a main stay of articles and TV shows about America: (such as Robinson’s Don’t Give Up On America).

To me, there are two [things] that are paramount. First, we must set missions with clear, achievable goals — not ones we’ll never reach.

In this environment, a government with a larger appetite for risk will be able to make some meaningful gains. China has taken this leap of faith.
Xi Jinping May Be Leading China Into a Trap (Brooker)

Bloomberg (archived), 9 June, 2021

What is the West’s fear about China? It is that Xi and the Chinese Communist Party will rewrite the rules of traditional economics and show that the power of markets, which is fetishized by institutions such as the IMF and the World Bank, is limited and only one of the many possible pathways to transitioning from a lower-income economy to a higher-income economy. This article crystallizes the terror that Xi’s recent non-capitalist reforms strike in the hearts of the traditional capitalists: Capital is not King; Governments that focus on Social Reform can succeed. Bloomberg Media is owned by Michael Bloomberg and is pro-capitalism and staffed by economists who believe that the West’s path out of poverty is the only path that exist and is viable. Their articles criticizing Xi and the Chinese government for not accepting the power of markets or institutions always provide an interesting perspective. For me, they show the height of the pedestal on which these journalists and economists place markets, even though they will freely admit that they are tools for development and ridden with many shortcomings.

This paragraph stood out to me:

If Xi succeeds in steering China into the high-income bracket without undertaking the institutional reforms that have accompanied the transition in other countries, then it would rewrite the rules of conventional economics and burnish the international standing of Beijing’s authoritarian governance model. Developed democracies have hardly demonstrated their superiority in this regard recently, having failed to reverse their own decades-long trend of widening inequality (even if disparities mostly remain far narrower than in China).

This rewriting of rules spells disaster for most publications, and is a common thread I have noticed in recent China coverage. Bloomberg is no exception.
China’s Port Shutdown Raises Fears of Closures Worldwide (Deaux, Li, Koh)

Bloomberg (archived), 13 August, 2021

World trade. I have found the topic to be confusing and intractable. How can the delay of a shipment in Philippines stop assembly lines halfway across the world? Now, I understand the basic contours of this concept: companies set up supply chains to procure materials from around the world (because they get it from the cheapest buyer) and assemble them at some central place; each component that they procure has its own supply chain. This article demonstrated the fragility and sensitivity of this system well. A port closure in China saw shipping volumes drop across the world. Shipping prices have been spiking since last year due to unrelated reasons and both bulk and container shipping prices were affected. This article is not very detailed in how the port shutdown affects the prices of real-life products; Nevertheless, it is a good primer. I’ll be on the lookout for another article which expands on the topics covered here.

Open Source Software Business Models

2021-09-15T00:00:00+00:00

A few days ago, AWS released version 1 of OpenSearch, a fork of ElasticSearch. OpenSearch is open-source software licensed under the Apache License v2. ElasticSearch’s creator, a for-profit company called Elastic Co. has been involved in a dispute with AWS for months now. The dispute began when AWS packaged ElasticSearch and started offering a managed service on their cloud platform. There seems to have been no partnership between AWS and Elastic, and this lack of a partnership has irked Elastic no end. This raises a question that others have attempted to answer: What is the business model for free and open-source software? Who should fund its development? Who should benefit from the profits made using that software?

I like free and open-source software (OSS). My rationale for using it is simple: It allows me to choose how I spend my time. I can decide what software packages to use, which version and for how long. If I don’t like a future version of a software, then I can stick with the version I like, until the time that security fixes are released for the version I have. I can decide the OS version I want to use and use it for 10 years¹ without having to install updates or upgrade my hardware because a future version of the OS slows down my computer. (Or phone.) This “decade-long end-of-life cycle” is important to me because I don’t want to spend any time installing, updating or upgrading operating systems².

I don’t have an ideological reason for using open-source software. The open-source ideologues and open-source software users are not one and the same. Most people in the former group are also in the latter, while a minority of the latter is made up of ideologues. Consequently, I don’t think that open-source software is the silver bullet to solve all problems. Proprietary software regularly solves problems much better than OSS can ever aspire to: IntelliJ is a set of proprietary tools for working with PHP that is very powerful and I use it at work when working with a large PHP codebase. It has a good plugin ecosystem and supports every process of the software development cycle. Photoshop and the Adobe ecosystem for editing photo and video files is another example of proprietary software basically taking over the world.

Another piece of proprietary software changed the nature of computing for billions of people: Windows is the most widely used operating system. It’s usage is widespread enough that most people (including me) started computing on a Windows machine at home or in school. Without a powerful OS such as Windows which was easily available on computers from major brands, fewer people would have access to a computer and the Internet today. Windows solved the access problem extremely well.

Open-source software is a huge beast. There are millions of projects with wide variety in the ways in which they are funded, maintained and the ways in which the project’s backers ensure its longevity. I will look at three models that are common.

No Business Model

The simplest business model is to not have any business model. Most projects on GitHub are not written with any aim to make money from them. Widely used tools such as ripgrep, notmuch and htop do not solicit donations³. The maintainers of these projects have been working on them for a long time but they do not expect any monetary gain from their work. Philosophical reasons for this apparently altruistic endeavor aside, this business model is popular and hassle-free.

While for-profit companies can start open-source projects with the intention of making money at some point in the future, community-driven projects (projects that start with a single maintainer or a few people working on it in their free time) are more common. This lack of a business model at the inception of a project makes it hard to implant it into the project at a later stage.

One such project which tried to put in a business model at a later stage was the company behind the Brave browser, based on Chromium and published by for-profit Brave Software, Inc., started out without any good business model. It has since tried to introduce “Attention tokens” and make an Initial Coin Offering, which was the thing to do in 2017.

A notable proprietary to “free and open-source” software transformation story is Microsoft’s Visual Studio Code editor. It has an interesting history. The proprietary software called VSCode was originally released in 2015 and seems to have come out of the Microsoft Visual Studio IDE project. In the subsequent years, Microsoft released most of the code behind VSCode, and published that as an open-source editor under the permissive MIT license. Microsoft maintains a distribution of the open-source editor which includes some proprietary assets but is free for private and commercial use. The Community version of the parent product (Visual Studio IDE - Community) is available for free download but can’t be used in Enterprise settings. This Enterprise version of VisualStudio IDE is the product that Microsoft sells to companies. (The product is similar to IntelliJ but not as versatile or modern as it sticks with technologies that are used to develop Windows apps, such as C# and ASP.NET.) Microsoft chose the “No Business Model” path for the open-source Visual Studio distribution.

For-Profit Companies Redistribute for a Fee

The original large-scale open source projects was the Linux kernel, the software that sits between the operating system and the hardware that it is running on. Linux became very popular and people spent large amounts of their time working on the Linux codebase. Eventually, a non-profit called the Linux Foundation hired the maintainer. This enabled the maintainer to spend all his time on maintaining the kernel and deciding the long-term roadmap of the project. The non-profit foundation is supported through donor contributions.

Most large companies use Linux in some form. Large scale cloud platform providers like AWS, GCP and Azure use Linux to deploy their datacenters and offer several services which are based on Linux and other open source projects to their users such as Google’s Compute Engines which offers virtual machines running Linux operating systems for Compute or Google’s Kubernetes Engine which is a managed service running the open-source Kubernetes container orchestration software.

These profits (or a part of them) don’t directly make their way back to the Linux maintainers, and the maintainers are okay with that because they are getting back enough compensation to work on the project full time and keep the project going⁴.

For-profit companies like Canonical (publisher of Ubuntu) and Red Hat (publisher of CentOS) have built open-source software on top of the Linux kernel. This software is free and open-source as well, these for-profit companies redistribute the free software for a fee.

Another recent open-source project is Wireguard. Wireguard can be used to create a secure Virtual Private Network (VPN) between any two devices which are on the same network (e.g. the Internet, a router inside a house or at an office). The maintainer began working on this as a side project and as a replacement for OpenVPN, the historical VPN standard. Wireguard also runs on donations from companies and individuals, in the form of money and hardware, as a way to express gratitude for the work that the maintainers are doing.

Tailscale is a for-profit company which sells a service that uses Wireguard underneath and allows users to create a network between all the devices that they own using Single Sign-On providers like Google and Microsoft to authenticate⁵.

Companies like Canonical, Red Hat, and Tailscale are not under any obligation to donate to the maintenance of the software that their businesses are based on. But most such companies which are redistributing free software for a fee, with added features, do donate back to their root projects. This generosity is one clear business model for open-source software maintenance.

This model has some conflicts of interests in the form of a donor asking for features to be prioritized based on their business requirements. Most open-source projects have some kind of private mailing list or offline meetup where the most frequent contributors decide the trajectory of the project and brainstorm about the roadmap and longevity of the project. I don’t hold any grudge against maintainers for this kind of non-public discussion, as I don’t believe that the public square is the right place to decide the road map of any open-source project.

Maintainers Redistribute for a Fee

Now, this type of business model is at the center of the AWS-Elastic dispute. Elastic is the for-profit company that maintains ElasticSearch, a full-text search engine that is popular in consumer-facing applications. Most search engines are either running ElasticSearch or running something very similar to it. This search engine product powers product search in e-commerce applications, timeline search inside social networks and full-text search in some blogs and text-based websites.

Amazon Web Services (AWS) is a Cloud computing platform. AWS offers several “Managed services”, services that have considerably lower maintenance costs due to their “managed” nature. These services are preferred by small software engineering teams as it frees up their time and enables them to focus on product development and rely on AWS for the maintenance of mission-critical software like the relational DB at the center of their operation or the search engine that powers their user’s search queries.

The core dispute in the AWS/ElasticSearch episode seems to be the use of the “ElasticSearch” term in the name of AWS’ managed service. Elastic offers its own products such as Site Search and App Search that use ElasticSearch under-the-hood and are sold with convenience features that make it easier to integrate them into real-world applications real-world applications. The AWS managed service and some of Elastic’s products are competing for the same part of the consumer search market.

A similar product, Elastic Cloud on GCP, is offered by Google Cloud Platform (GCP), a competitor of AWS. The key difference between the two managed services seems to be that GCP and Elastic entered into a partnership, while AWS struck out on its own to create the product that it wanted, without any consultation with Elastic.

Elastic did not like what AWS did at all. In fact, it annoyed them so much that they decided to change the license that ElasticSearch is distributed under:

It also protects our continued investment in developing products that we distribute for free and in the open by restricting cloud service providers from offering Elasticsearch and Kibana as a service without contributing back. – https://www.elastic.co/pricing/faq/licensing

This business model is a dicey one and ridden with conflicts of interest. Elastic was not really on board with free and open-source software from the get-go, especially if the software was deployed on a large scale and could possibly poach the market share that it held through its own managed service products.

The change in license was the right thing to do, as it clarifies their intention behind maintaining ElasticSearch: The license should embody exactly how the maintainer wants users to treat their software. Their decision to delay this change this long seems to be based on their false conviction that few people will be able to figure out how to run ElasticSearch at scale without Elastic’s help or that a competitor with as many existing customers as AWS will not step on their toes, due to fear of retaliation.

AWS’ decision to fork ElasticSearch to continue offering the managed service and a search engine that is modern and innovative was also the right one. The fork, OpenSearch, has had a good start, reaching version 1.0 in 3 months. (The project began at a stable ElasticSearch version anyway, so this might not be as big a milestone as reaching 1.0 is for other projects.) As the two projects diverge, it will be interesting to see which one outpaces the other in terms of development and new features.

Elastic has nothing to do but work on ElasticSearch, whereas AWS is a cloud-computing company, running one service based on this fork. I would like to see AWS succeed in maintaining the fork and continuing the addition of interesting features long into the future. But the fact that this will not be their only priority seems like a risk for early adopters of OpenSearch.

Another important question which the product will have to deal with down the line would be whether they would like to maintain compatibility with ElasticSearch in the configuration and query language. AWS (the business) would prefer maintaining this compatibility. The project maintainers might have different opinions in the interest of the future health of the codebase.

Edit 2021-09-15: As I was editing this post, ElasticSearch decided to update their Open Source clients to stop allowing connections using their maintained clients to OpenSearch clusters. Amazon has forked Elastic’s search clients and plans to maintain both the search engine (OpenSearch) and its clients in several programming languages.

To give these users a clear path forward, the OpenSearch project will add a set of new open source clients that make it easy to connect applications to any OpenSearch or Elasticsearch cluster. These clients will be derived from the last compatible versions of corresponding Elastic-maintained clients before product checks were added. – Keeping clients of OpenSearch and Elasticsearch compatible with open source

Elastic’s decision to prevent connections to OpenSearch has increased Amazon’s maintenance cost manifold. I hope that this move does not deter AWS from their open-source software crusade.

I am using Ubuntu 18.04.5 LTS. I installed it in 2019 when I bought my current laptop and I intend to use the same OS for the remaining 7 years until its End-of-Life. ↩
This is a mildly controversial point-of-view within the software engineering community where computers are often equated to Lego sets; installing new operating systems, window managers, upgrading and tweaking with the hardware of your computer is the backbone of the hacker mindset. ↩
Written July 2021. ↩
In 2018, the maintainer of the Linux kernel, Linus Torvalds, had a compensation of $619,472 according to a disclosure made by Linux Foundation. ↩
Wireguard’s in-built mechanisms for connecting computers is technical and it is not easy to walk others through the set-up, while Tailscale relies on easy-to-understand Google and Microsoft accounts and email addresses to connect computers. ↩

Suggestions For Better Government Services

2021-09-14T00:00:00+00:00

I have dealt with government services in multiple places now, and I believe that I have found the common pitfalls that service providers often fall into. This post is not a rant. There is already a lot of documented evidence of governments that were unable to deliver services well, frequently leading to consequences ranging from mild annoyances to tragic outcomes like death: Regional passport offices, post offices, nationalized banks like State Bank of India and Regional Transport Offices (RTOs) in India, Immigration centers in Japan, and healthcare.gov and DMV in the US. This post lists some of the simple enhancements that service providers can implement at low to medium costs to improve the consumer’s experience.

The Internet Is A Powerful Communication Tool. Use It Wisely.

There is a line between using the Internet to disseminate information to consumers, and designating the Internet as the only medium through which service can be obtained. The latter is ridden with problems emanating from unequal access to the Internet and is not a good idea. On the other hand, the former is a noble cause and service providers should put information about the services that they provide, the times that they are open, and the amount of time that each service is expected to take on their websites. The websites should be easily accessibly in the commonly spoken regional languages¹.

The gold standard is the list of procedures on the Immigration Services website of Japan. These services can be availed by foreigners in Japan. The website is available in Japanese and English, and each procedure has the essential things that people want to know about it:

List of required documents
Standard processing duration
Where to go, timings that the office will be open for
Fee that the applicant will have to pay

This is the absolute minimum that the applicant must know before they go to the center. To avoid multiple trips, maintaining an accurate list on the website will reduce the administrative load of the service provider (people will not have to be told what to bring or told to come back later) and will make the applicant’s life immeasurably easier. The reduced anxiety and stress that is felt by applicants is objectively a good thing.

Use Token Numbers Instead Of Queues

Another common image is a line of people who are standing in a queue outside a government office. In pouring rain or under the hot sun, the line of people remains unchanged; while the service provider is sitting inside an air conditioned office under a concrete roof, with a hot beverage on their desk. The injustice is painfully obvious, and this cruelty serves no one. This system is open to caricature and Kafka does government caricature better than anyone else. (Suffocating rooms inside bureaucracies are as common in his novel, Trial as primary colors in a child’s landscape painting.)

Token numbers are the obvious solution to this problem. The service provider’s office should have two doors: one with a simple reception where monotonically increasing token numbers are dispensed to people, along with the time at which they should show up. The other door leads the applicants into the actual office where they can apply for the document or the service that they came for.

This solution is fairly common in Indian banks in metro cities where almost all the business in a bank can be done by getting a token when one enters the bank, waiting (while seated) in the chairs and benches around the branch, and watching a television screen that displays the token number that is currently being serviced.

The implementation of such a system is extremely cost-effective. Both the economic cost and the administrative cost of setting up such a system is minimal. I would go so far as to say that a motivated bureaucrat with the appropriate amount of political capital within their department can turn around the functioning of any government office and increase the efficiency manifold through such a system.

Further, this kind of system is open to data collection. Once the system is working smoothly, bureaucrats at higher levels should be able to easily collect data about what kind of services are taking up too long to dispense and how these processes can be improved to reduce the burden on government workers.

For this suggestion, I will offer a caveat. The obvious next step to this “Token numbers” system is the one which tries to allocate appointments to people through an appointment system on the Internet. This increases the benefits for the applicants manifold. But for the service provider, implementing such a system is extremely high cost. The economic cost of implementing such a system is quite high. The initial administrative cost of starting this implementation is nearly insurmountable in most government settings where short-term results are key to career advancement. A long-term project which starts with a tendering process to find a digital vendor to build this appointment booking portal will almost always end in tears².

India’s Passport Application Process went through a revolutionary change between the first time I applied for a passport in 2005 and the most recent time I applied for one in 2015. The system went from a standard, Kafkaesque government office experience (complete with mezzanine ceilings, wooden benches set against concrete walls, and the essential profound, suffocating despair) to a modern office building run by a private company that prizes efficiency with a single government functionary who takes the final decision about whether the passport should be issued or not. One prominent part of this new system is the appointment booking system which allows applicants to fill out all their information online and then reach the passport office at the appointed time for their appointment. But there’s a major problem in the system.

The system is based on government information and does not clearly explain what documents are required from each type of applicant. The list of documents change depending on your age, your phase of life, and whether you are applying for a new passport or a renewal of your existing passport. These rules are extremely hard to parse for human beings, but they are extremely easy for a computer to navigate given the right number of inputs. In any case, the lack of this basic part of the system ended up in an extra trip to the passport office for me because the documents that I had were not sufficient.³

Compassionate Service Providers And Applicants

The dehumanization of healthcare is a scourge in our society: the delivery of bad news to patients has become another routine task. Medical dramas like Grey’s Anatomy and House M.D. frequently feature compassionate healthcare providers in lead roles. A society-wide agreement about dehumanization being bad for society is yet to come. Given that cliffhanger of a decision that society is (as yet) unwilling to make, service providers inside governments can lead the charge by employing a humane and compassionate workforce, filled with people who are willing to go the distance for applicants who are (always) helpless and (totally) dependent on the whims of the service providers.

This piece of advice applies to applicants as well. The spectacle of a frustrated applicant screaming at or threatening a government functionary is not uncommon. The functionaries are often themselves powerless, employed at the clerical level, and were probably in the wrong place at the wrong time, leading to this humiliation. Here the applicant must also understand the wheels that are in motion behind the scene. As a cog in the system, neither the applicant nor the government employee can effect any large change. It would be wise for these two groups to stay together and strive to implement changes that would benefit both groups, that so often seems out of reach, by converting their shared anger and frustration into well-researched criticism and directing it at the people who are setting up the systems rather than the people who are simply following handbooks which are given to them.

The primary misalignment in all government services is that the application process is set-up to prevent a bad-faith actor from going through the process without tripping any of the wires, even though the majority of the applicants that go through the process are good-faith actors, who do not intend to hoodwink the government. In higher-income economies where economic tracking of the citizenry is accurate and multi-pronged, governments are confident in their ability to identify bad-faith actors without these exhausting roadblocks in routine application processes. In lower- and middle-income economies where economic tracking is weak and ridden with loopholes, the government’s trust in its citizens providing accurate information is low and this is the root cause of the high administrative burden to acquiring services in these places. But even this burden can be made easier to bear through low-cost solutions and governments which implement such solutions will find much favor among the residents of their jurisdiction.

Note here that I am not recommending service providers to provide this information in a million different languages. The additional cost for building this kind of a multi-lingual website is considerable. The benefits that are gained are marginal. Services like DeepL.com and Google Translate are freely available and easy to integrate into most browsers even for normal users. ↩
For a recent government service, I visited the website of the Tamil Nadu Registrar Offices. This portal has an appointment system and is complicated enough to warrant a user manual for each process that is offered on the portal. The user manuals are excellent with screenshots and instructions about each step in the application process. But creating them probably took as long as building the website itself. This extra cost is unnecessary and should be done away with in initial implementations of the token system. ↩
A common parable about going to Indian government offices is that the officer there will ask you for the one document that you don’t have with you. ↩

Monthly Recommendations (August 2021)

2021-08-31T00:00:00+00:00

This month’s theme is art and the artist. I chose this theme because events in my life have reminded me of the value of the abstract and incomprehensible. As places around the world open up under varying levels of restrictions, the contemporary locations and events were art is showcased, namely, cinemas, theaters, live concerts and art museums, are returning to their earlier glory and posing an interesting question to a subset of the population that has the luxury of deciding whether they want to attend: How much risk are you willing to take to see an artist perform live? This month’s list begins with an art critic who argues that our appreciation of art from the times when life expectancy was low is bound to increase, owing to our own deeper experience of mortality through pestilence. The list also includes articles about the perpetual search for the line between art and the artist and about the limits of on-screen fantasy and escape, when the spectator’s real life is in disarray.

Out of Time (Schjeldahl)

New Yorker (archived), 13th April, 2020

I like going to museums of all kinds: art, natural history, national artifacts, museums memorializing disasters. A quote that I read in a museum in Taiwan 3 years ago has remained with me: “If we seek to understand the present and predict the future, the past is our only guide.” Schjeldahl, a venerable art critic, writes about what differentiates the experience of seeing art in person and seeing it on a computer screen or on a virtual tour. He also writes about the overwhelming presence of death in ancient art; mortality was experienced an integral part of life in the years when life expectancy was low and the reasons for death included even the most mundane causes such as the common cold, compared to current times when it borders around the periphery during normal times and when one thinks about death only as the consequence of severe disease or disaster.

He also writes about our return to art museums, a topic that I am particularly interested in. I have been able to go to art museums and live concerts over the past year, during the brief period that the pandemic let up in Japan in September and October 2020. As the relaxation continues and more people return to art museums, I wonder if their tastes have changed and if we end up being more enamored with the art of the “Ancient Masters” from having suffered through daily reminders of the fragility of life in the form of case numbers.
Fearless Pianist (Ross)

New Yorker (archived), 18th May, 2020

This is the profile of a German pianist who is also a political activist. It delves into the question of separating the Art that is produced by the artist and the artist themselves. This question has haunted both artists and spectators for a while now. Over the past year, with the accusations that sprung up against comedian Louis CK and the new documentaries about musicians R Kelly and Michael Jackson, the question pushed through into the mainstream and reached a population that had remained insulated. The ultimate decision about whether the connection between art produced by an artist and the artist’s actions and opinions must be (necessarily) left up to each individual. This article poses this question forcefully in the instance where the artist’s political opinions don’t align with your own. It made me think about the artists that I adore and how I would react to similar situations that involved them. I think that questioning and clarifying your beliefs before the situation arises is the only way to stay sane and continue to patronize their art, respect the artist and defend yourself to others. Kendrick Lamar poses the same question more succinctly in his song Mortal Man from the album To Pimp A Butterfly:

If I’m tried in a court of law, if the industry cut me off

If the government want me dead, plant cocaine in my car

Would you judge me a drug-head or see me as K. Lamar?

…

When shit hit the fan, is you still a fan?

When shit hit the fan, is you still a fan?

– “Mortal Man”, Kendrick Lamar
Hot and Toxic (Goodyear)

New Yorker (archived), 16th March, 2020

Following along on the separation between art and artist, this profile of the artist Wolfson focuses on the provocation that he sees his art as a vessel for. In this case, the artist’s personal beliefs are not reflected in their art. In fact, the art is designed specifically to incite the spectator into thinking about … something. Being “edgy” has been an asset for artists like Pollock, who thrived on abstract art with no definite interpretations. Everyone drew on the canvas that Pollock had already drawn on; albeit they started with a slate that was packed with material and guided their thoughts through particular paths, not letting them wander aimlessly. Wolfson’s art is being made in the age of the Internet and career ending mis-steps can come out of the artist’s inherent belief about the role of art in society. Is it safe, then, to step outside the mainstream and create provocative art?
“All the Natalie Portmans” and The Limits of Hollywood Fantasy (Cunningham)

New Yorker (archived), 16th March, 2020

I liked this article on two levels. The first level was for the content: It is an amazing review of a play which features a young teenager who dreams of being a screenwriter and idolizes the actors who have made it big in the 2000s. The second level was for the effortless references that the author draws on about Portman and her role in building a fantasy. She mentions several of Portman’s movies and her recent Oscar appearance. Coincidentally, I happen to be reading Dyer’s Heavenly Bodies, an essay about how stars are made and what society really looks for in them.

P.S. This article introduced me to Dior ad campaign that stars Portman:
Chain Me Up: Harry Houdini and The Art of Escape (Denby)

New Yorker (archived), 30th March, 2020

Houdini is a household name today. What was he escaping from? Was there some meaning to his art and work? Or was it meaningless and simply the culmination of Houdini’s personal desires? Houdini’s defiance of authority figures and the police in particular takes center stage in his work. He demonstrated the crass, anti-authority culture of rebellion through an art form that captivated everyone. But what he did was not sleight of hand or magic; his escapes were real acts of strength backed by an endless amount of preparation and presented with a complete lack of showmanship, which has traditionally been the mainstay for magicians.

Power, Age and Maturity

2021-08-22T00:00:00+00:00

At first glance, the minimum age requirements on various powerful positions seem unnecessary. Why should a meritorious individual be restricted from doing something that they would be good at simply because they are too young? Over the past few years and especially over the past few weeks, I have understood one of the reasons for these requirements: Power should be handed to people who have the philosophical backing to handle it; they should be grounded in the understanding of their role; they should not exploit the people who are newly under their power; they should know where the boundaries of their power are and where it is being used for exploitation. Age is a proxy for this maturity. Older people are assumed to have this maturity and younger people are assumed to be naive, immature and ignorant. This is never really true. Age is a bad proxy for maturity. It fails in both directions: Older people can be less mature (and younger people more mature) than their age would lead you to assume. Most of these failures are not consequential as the power that these old people hold in society is insignificant. But the power that they hold in your personal life can be considerable and this is what I will dig into in this post.

“Power corrupts …”¹ is one of those phrases which is so popular that one needn’t even repeat the whole quote. I did not quite understand what it really meant. Sure, I understood the overall concept of it: Someone gets some power somehow; they become arrogant and unreasonable; the people around them try to take them down and are sometimes successful. But I did not understand the real-life consequences of it before. Now that I understand what the “corrupts” in that quote really means, I can see some past instances in my life where this principle was at work and where people had been corrupted by the power that they believed themselves to be holding.

Universities Are Small Ponds

The most memorable instance is from college. Professors in my college had a lot of power. They were instructors in courses where they had complete control over scoring, who passed the course, what their grade was. Their hard power was taken to extremes during the Bachelors and Masters project presentations which happened at the end of the fourth and fifth year of study. During the final year “Grand Viva,” there is a positively absurd air around them and their questions, as memorialized in this brilliant piece.

The ages of these professors varied. But most of them had only been in academia: Bachelors from a college in India, Masters from somewhere in America, Ph.D. in America and Post-Doctoral Fellowship in America or Europe. In their early 30s, they had turned up as an Assistant Professor with no real experience to speak of. They were big fish in a small pond. They did not know the size of the ponds that were out there, and they had no interest in finding out, for they had chosen to hole themselves up in a remote location inside an echo chamber where no one dare challenge their views.

As soon as they arrive, they were given a large amount of power. Some of them were made “Faculty Advisors,” a (mostly) meaningless position which required them to sign documents for a few students doing bachelors courses. While this position’s requirements were nearly non-existent, this process of getting your advisor’s signature on some document would strike dread into the heart of even the most ardent believer in the rot that are bureaucratic processes. My experience was perhaps slightly better than the worst ones and worse than the average experience.

My advisor would sign papers from 4:30 pm to 5 pm on weekdays. (What Kafka reader would expect government servants to work during the weekend?) During all other times, the papers would have to be dropped into his letter box and he would sign them at his convenience, which would generally take a couple days.

On one occasion, I had the (mis)fortune of having to get a signature by the end of the day due to a quirk in the course registration process that required each document to go to the same person twice and for them to give their approval once on paper and once on the computerized course registration system. I arrived at the advisor’s office at 4:40 pm, well within the 5 pm time limit. I knocked on his office door, which was at the center of a deserted fluid mechanics laboratory. After a few seconds, the door opened a couple inches. I could barely see his face and perceived a pair of piercing eyes which appeared annoyed at my presence outside his office door.

I had the paper in my hand (I had had the presence of mind to waste no more of his precious time than I needed to.) I started explaining what I wanted: “I need your sign to register for …” Exasperated with my attempt to explain the situation to him, he cut me off and said “Okay.” He proceeded to come outside the office and closed the door behind him. (He must have stepped out 1 feet or so and there was no invisible burglar hanging around behind my back. Perhaps he was ecologically conscious and did not want power to be wasted through an Air conditioner which kept his office at a comfortable 23 degrees, as the outside temperature was around 35 degree in the hot, summer weather.) He snatched the paper from my hands, placed it on a nearby table, signed it near the bottom, handed it back to me and disappeared back into his office. The quick glance that I caught of his office did not show me anything remarkable: It was a bare office with a lot of empty space, a computer, a table and chairs on both sides.

I went through this routine a few times during my 4 years under him as my advisor. There was no call for this kind of behavior. In his early 30s, he had clearly not understood how to handle power (or how to be a considerate human being.) There was no sign that he wanted to, for he seemed to enjoy this perverse dance and practiced it with everyone. In a college with tenured professors, there are no avenues from which professors might be challenged or evaluated for their behavioral quirks. There is no Human Resources Department that I could have appealed to for help. There was no Manager whom I could have sent an email to about this absurd behavior. Their positions are extremely secure and no professor has ever been let go. Even the most egregious characters, those who push the limits of even the most eccentric understanding of the state of “normalcy”, hang around for several decades.

88% Believe That Respecting Elders Is Very Important To Being Truly Indian

Another common pitfall of using age as a proxy for the ability to handle power is common inside families. In this setting, age is a pure proxy: There are no qualifying or redeeming qualities that one might appeal to. Nothing else is even worth considering as a criterion for absolute knowledge.

A recent Pew research center report confirmed this finding: 88% of the respondents said that “respecting elders is a very important part of being Truly Indian.” Due to the (rather unfortunate) linearity of our experience of time, it is always true that there will be people who are older than you inside your family. How often you meet them hardly matters as long as you are younger than them.

These are some of the common phrases that your relatives will resort to when you challenge their (often antiquated) beliefs with rational counter-arguments based in history and your understanding of how society has changed since the time that those beliefs were formed or found to be fashionable.

You are young and don’t know how things are done
You are young and don’t know anything
You have lived away from your native place for too long and have no idea how things are done here
You are trying to be different just for the sake of being different and you don’t actually believe in the things that you are saying
I am older than you and you should not talk back to me

These phrases are mildly amusing to me. At 25, I have the devil-may-care attitude typical to people in their 20s. I have the temperamental luxury of using a combination of sentimentality and rational thinking when deciding things, heavily biasing towards the latter as much as possible. Any combination including rational thinking is anathema to the power-hungry, old person. Their power comes from a combination of people respecting elders (extremely common) and being sentimental (not uncommon).

The last phrase in the above list is worth examining some more. Children are often trained to not “talk back” to elders when they are being berated and to instead nod their heads and move along and do as the elders please. This turns out well for most kids; a minority of them seem to act out.

Obedience is conflated with tacit acceptance of the elders’ points-of-view. Obedience is further conflated with respect and good manners. I disagree with both these conflations and I believe that they are part of an insidious scheme to keep communities together.

(The conflations between obedience and acceptance, respect and manners is infuriating. I believe that I can have a respectful debate without accepting my interlocutor’s point of view, respecting them as a person with their own set of views, and staying within the boundaries of good manners. If it is truly impossible to do this, then surely the understanding of “manners” must be comical and absurd.)

People asking uncomfortable questions have introduced grievous rifts in religions and civilizations before. One need only look to the rise of Lutheranism (Protestantism) and the split of Christianity into the two modern sects of Catholics and Protestants. Luther’s questions were an attempt at challenging a practice that aimed to extort money from the followers of the Catholic Church. Even the Catholic Church, an institution as old as they come, could not provide a satisfactory answer and thus the revolution began. How the Catholic Church must wish that Luther had been taught never to question institutions that were significantly older than any human alive.

This historical weakness of civilizations to questioning individuals and the ultimate refuge of sentimentality is embedded inside our appetite to convince children not to “talk back” to elders. The same teaching also applies to people who happen to be young at the current moment.

I will attempt to boil down what I want to say: Believe in something and Act with conviction. As for what others will say when you do so, I will take a few lines from Seneca’s Letters:

You needn’t believe the chatter of the people around you: there’s nothing in all this that’s evil, insupportable or even hard. Those people are afraid of these things by a kind of general consent. Are you going to feel alarm at death, then, in the same way as you might at some common report? What could be more foolish than a man’s being afraid of people’s words? My friend has a nice way of putting things when he says that to him the utterances of the unenlightened are as noises emanating from the belly.

– Letters From A Stoic (Seneca)

This was first said by Lord Acton and his views on why power always corrupts are summarized in this essay. ↩

Monthly Recommendations (July 2021)

2021-07-31T00:00:00+00:00

This month’s theme is cities, islands and the places that we live in. Few events in my life have triggered as much introspection about the city that I live in and the places that I go to as the ongoing pandemic. Over the past year, people have left cities and retreated into the countryside, finding that their work remains unaffected even if they don’t go to offices and don’t meet their coworkers in real life. But how should we account for the damage to the city’s fabric because of these exits? Optimists continue to believe that things will go back to the way they were and that there will be no long-term impact due to this aberration that we are living through; I count myself in this group, foolhardy as it might be to expect a future identical to the past. Revolutionaries and activists see the leveling of lifestyles between geographies as a step towards equality, a spiritual return to the mixed socioeconomic class communities that were fixtures of towns and villages up to the late 19th century. In an effort to explore this ongoing change more fully, this month I recommend a photo essay that documents a nurse’s daily life, an article about the differing paths for social change depending on the average income of the district that you live in, and an essay about how the place that you grow up in is embedded within your personality.

A City Nurse (photo essay) (Remnick)

New Yorker (archived), 4th May, 2020

As the COVID-19 pandemic started taking hold and hospitals around the world, starting with Italy and then New York and then pretty much everywhere at once, dealt with a huge number of hospitalizations, politicians started expressing gratitude to essential workers at the beginning of all their speeches. Indian leaders started extolling these people as “warriors.” These rhetorical changes did not go hand-in-hand with policy changes that might alleviate unprecedented pressures on underfunded healthcare systems. This photo essay details the life of a nurse who went through the worst parts of the pandemic in New York. This was a powerful reminder that the people working inside hospitals were put on a pedestal, but nothing was done to protect them or to improve their lot by politicians and society at-large. The gratitude expressed never turned into collective action (wearing masks, social distancing) or policy action (building large field hospitals, preparation for subsequent waves). Cady Chaplin, the nurse profiles in this article, reacted to the platitudes with matter-of-fact stoicism:

Many evenings, at seven, Chaplin can hear the cheering and honking, the nightly tribute to the “essential workers” who are keeping the city alive. The sound often makes her tear up with gratitude, but she is wary when she hears platitudes about the “heroic” work of health-care professionals. She doesn’t want to be glorified all of a sudden. “This is what we trained to do,” she says. “This is what we do. That was true a year ago, and it will be true a year from now.”
Abundance of Caution (Gopnik, Montgomery)

New Yorker (archived), March 30, 2020

Working patterns have changed over the past year, and if you live in a city, you might have asked yourself what it is about cities that attracts and retains people despite there being no requirement to live there anymore. This is a story that is still being written; cities might well become less popular and end up as abandoned jungles of concrete-and-glass buildings with empty offices. But there is very little indication that that is the direction we are headed for. And so, when people return to cities from their present hiatuses (either from inside their homes or from a completely different place), will the fabric that bound it together remain unchanged? Or has it been damaged by the temporary abandonment? The message in this article is pretty clear although it is not explicitly stated: When the rich-and-wealthy abandon the city where they made their fortune, people notice.
Men Waiting for a Train (Biespiel)

New Yorker (archived), 18th May, 2020

This is a poem and it is surprisingly relatable¹.
My High-School Commute (Jost)

New Yorker (archived), 16th March, 2020

This article, written by SNL Weekend Update’s co-host Colin Jost, is about how the cities that we grow up in are special to us simply because we were there during our formative years. I catch myself waxing nostalgic about how great the city that I grew up in was, even though I never really knew much more of it than the neighborhoods that my house and school were in. Stranger still, I sometimes vehemently defend it against any attack from people who lived there as adults and saw much more of its people and their idiosyncrasies. I doubt that this feeling will ever change for me and this article helped me understand its roots.
Juul Finds Hell Hath No Fury Like an Army of Really Rich Parents (Etter)

Bloomberg Businessweek (archived), 14th May, 2021

The wealthy and the well-connected inhabit a world where social change comes from private activities much more frequently than from the traditional route of dreary city council meetings and convincing elected officials. This excerpt from a new book about the American company, Juul, details the campaign against its “vaping products”, a cigarette-like product that emits white smoke and delivers nicotine instead of cigarette’s mix of tobacco and nicotine, in one of America’s wealthiest districts. In these parts, acquaintances from the upper deck seats of a college football stadium can lead to increased pressure on a private company from its regulatory body. Things get done, undoubtedly; whether the benefits of these changes will reach people outside the wealthy parts is not their concern. This article reads like a thriller and introduces a train of interesting characters, each armed with a roster of powerful connections.

This is a change in format. I generally recommend only articles that are of medium length. I also don’t know much about poems, so I stay away from them for the most part. This poem was particularly impactful and evocative and thus, finds a place on this month’s list. ↩

Let's Talk About Facebook

2021-07-25T00:00:00+00:00

American President Joe Biden recently made a sensational comment insinuating that Facebook’s permissive stance towards vaccine misinformation was “killing people.” This sensational attempt to blame Facebook for his administration’s inability to hit vaccination campaign targets left a bad taste in my mouth. I could not quite understand how he could credibly blame a private company, while he himself sat at the helm of the Federal Government. It reminded me of an exchange from the TV show West Wing in which the president says, “School boards and local elections are where true governance takes place.” In the TV show, the president’s comment is used to show how unhinged and paranoid the President had become and how he had let personal enmity cloud his judgment. (His advisors point out to him later that he has bigger fish to fry and can’t get involved in these local elections or swing them using his pulpit.) In the real world, Biden has used this technique to create a scapegoat for his administration’s failings.

From the time that Facebook came on the scene, someone has been completely in love with it, while someone else has been incensed at its existence. I believe that Facebook is polarizing because it demands something radical of the people who believe that they have some understanding of the world around them: Adapt to something truly new. This demand is rarely made by an ill-defined private entity with a misunderstood product and a (sadly) robotic CEO who can’t quite express what his goal for the product is. (When I watch Zuckerberg talk about his company, I wish he would just say that all he wanted to do with Facebook was build a cool engineering product and make a lot of money with it.) A demand like this one is not novel. But in the past, these demands were made by revolutionary, charming CEOs with well-understood products.

Two similar adaptation demands that come to mind are the ones made by Ford, who pioneered the auto-assembly line and increased auto-manufacturing productivity many-fold, and Steve Jobs, who pioneered the iPhone and insisted on every human being’s fundamental right to have a computer in their pocket. They insisted that society should live with these improvements, however uncomfortable they made the old guard of their time. The iPhone, in particular, was marketed by a sometimes-crazy, sometimes-revolutionary founder. These two adaptations became a part of the mainstream very quickly.

Facebook’s trajectory was similar: millions of users adopted it across the world. And yet, no one quite knew what to make of it. What happens when a misunderstood product becomes wildly popular? I think the answer to this question is the key to understanding where Facebook is right now. Facebook is on the cusp of becoming a scapegoat for spreading misinformation, is waiting for the other shoe to drop on antitrust allegations in the US, and is facing a mutiny among its employees, whose political beliefs are not representative of those of its user base.

Before proceeding, I will put my cards on the table: I don’t use Facebook anymore. I used it a lot from 2010 to 2014. I used it much less from 2014 to 2016. And I stopped using it around 5 years ago, when I was in my third year of college, because I could see that it was making me actively agitated. (This was the period in my life when it seemed like everyone around me was landing an internship or a conference talk and going abroad for the first time; while I was going to Bangalore to intern at a start-up no one knew much about.) I have a libertarian view of the usage of technology and the Internet. I think that people who like to use Facebook should use it. I don’t believe that paternalistic entities (like the government or a Netflix documentary) counseling people about the evils of social media is going to help one bit.

Now, let’s talk about Facebook.

First off, the most thorny question of all: what is Facebook?

Facebook is a product which allows people to post text and images, add other people as their friends, follow celebrities and news websites who post content, and see content posted by their friends and the pages they have followed in a constant stream called the Newsfeed.

That’s a mouthful. I tried to boil down Facebook’s core feature, Newsfeed, that also happens to be the most controversial. The other parts of Facebook are widely accepted as useful (or at least not harmful enough as the Newsfeed): messaging, games, groups, etc.

The Algorithm

The things that show up on a person’s Newsfeed and the order that they are displayed in is decided by “the Algorithm”. This constant reference to the algorithm is misleading. The thinking around this algorithm goes something like this: Facebook has a system where they can specify the weights for each of the pieces of content that a person is eligible to see. (These pieces of content enter this list because they were posted by the user’s friends, friends of friends, a page they follow, a group they are a member of, or because a recommendation algorithm predicted that the user will like said piece of content, among others.)

I doubt that Facebook’s internal systems are anywhere near that simple or easy to customize. My prediction is that the algorithm is a combination of handmade heuristics and one or more machine learning models, which have optimized to move some metric in the desired direction. (Common metrics could be of the form: Maximize the amount of time a user spends on Facebook; Maximize the number of comments that they post in a given time span.) Each machine learning model would try to classify the user into one of several buckets that it has created based on training data. Then, for each piece of content that it sees, it would decide how likely that piece of content is to move its goal metric in the direction that is desired. After this calculation, it would present the pieces of content to the user in an order of decreasing likelihood. (This is all complete speculation based on my experience with similar recommendation algorithms and my understanding about how such a system would be designed if it were to be designed from scratch. Facebook probably does not have such a simple, easy-to-understand design, because the system was not built from scratch, rather, it was built in an iterative fashion.)

By simplifying this possibly very complex system into “the Algorithm”, administration officials and activists can effectively use Facebook as a scapegoat for pretty much anything.

Surgeon General Vivek Murthy specifically calls on companies to redesign their algorithms to “avoid amplifying misinformation.” He also suggests that they build more “friction” into sharing functions that urge users to rethink whether to share a post containing false information. – Surgeon General calls on Social Media Companies to Curb Misinformation on Their Platforms

The level of control implied in the above suggestion shifts blame effortlessly. It paints Facebook as the private corporation which is unwilling to take steps to fix its algorithm, implying that the only thing lacking is Facebook’s desire to make such changes. The nuances of whether such changes are possible and how effective they would be is missing from this characterization.

The content posted by users on any platform is diverse and it is impossible to classify something as “containing false information”. The Internet will confirm your beliefs and your stand on any topic. There is very little objectivity remaining in the topics that have become part of the common discourse today.

I am putting aside the philosophical debate about what misinformation even is. A few examples:

Is something that is reported on a major news network misinformation?
Is the photograph of a report published in the New York Times misinformation, if the original report had a material error and the photo was shared without the subsequent correction?
Who is the arbiter in these cases? If it is the judiciary, is that branch of government independent enough from the executive to say that the executive was spreading misinformation?

The suggestion to make it harder to share “false information” assumes that Facebook is an omniscient entity which can classify something as false. Softening the definition of misinformation to “widely believed to be false” is not helpful because we are stuck with the same classification problem. Technological solutions are not the root cause of the misinformation problem. The root cause is not knowing what misinformation even is, until something becomes popular enough to warrant further investigation, a fact check, or public uproar.

Private Corporations are not Governments

I think this needs to be repeated: Facebook is a private corporation. Milton Friedman, the economist behind shareholder capitalism, insisted that the only responsibility a private corporation has is to its shareholders. i.e. Corporations like Facebook should do whatever it takes to increase the company’s value to its shareholders, within the boundaries set by law.

The “social responsibility” of corporations was always a sideshow. It is one that corporations are compelled to engage in when they come under activist pressure. Some notions of responsibility have been built into corporations through the “Stakeholder Capitalism” model, which claims that corporations have responsibility to parties other than their shareholders such as the environment and society. This notion has not taken off and I remain unconvinced that this approach will succeed in the long term. Forcing private corporations to do something other than make profits and increase shareholder value seems strange and counter-productive. There are other types of organizations who are engaged in this kind of activity: Non-profits look out for people who don’t have a voice. Activists talk to lawmakers about agendas that they care about. Private companies are not really supposed to be activists or anything other than profit-making corporations.

The ultimate non-profit is the Government. Governments have always been about everyone. (“Everyone” as in the people who are being governed and not being deliberately excluded due to discrimination.) One of the government’s jobs is to put the right amount of regulation in place, so that innovation continues unhampered, while citizens can live full, satisfying lives. When private corporations become as big as Facebook and Google, the government’s power does not simply vanish. The power held by governments is special and can not be taken away by anyone else.

Any country’s government can legally require Facebook to stop misinformation on its platform through these steps:

Enact legislation to make Facebook liable for the harms caused due to misinformation shared on the platform

That’s it. There is just one step to that process. Once the government starts finding pieces of misinformation, identifies the tangible harms caused by it, and holds Facebook liable for these harms, it is in Facebook’s interests to stop the spread of such misinformation. There is no “choice” or high-minded talk of “social responsibility” here. Facebook’s shareholders will not stand idly by as governments impose hefty fines on the company for not cracking down on misinformation. Facebook’s executives will be forced to act (or pretend to act or curb the government’s enforcement action).

The realist politics watcher will instinctively oppose this idea: Facebook pumps in a huge amount of money into the campaigns of people who become part of the legislature and no politician will enact legislation against their top campaign donor. This problem is not without a solution either: Campaign finance reform. Anyone can donate unlimited amounts of money to any candidate and remain completely hidden through organizations which don’t have to publish their donor lists. This was proven by Stephen Colbert who did the actual legal paperwork to establish such an organization and prove that it was possible. (The resulting TV show segment was also funny.)

The Convenient Scapegoat

Comments like Biden’s recent “They are killing people” are mind-boggling to hear. Mainstream news outlets have been peddling vaccine misinformation¹ and are arguably perceived as more reliable than posts by random anti-vaccination campaigners on Facebook. Given this disconnect between the various sources of misinformation and the government’s focus on Facebook, it is important to analyze the timing and the possible ulterior motives for this zeroing in.

Misinformation on Facebook has been going on for some time now and anti-vaccination misinformation has been spreading for a really long time. Savvy political campaigns, such as the one run by Donald Trump or those run by the Republican Party’s candidates during the Georgia run-off elections, have used the Facebook ads platform to peddle misleading information about their opponents. These ads were disguised as political advertising, and Facebook did not take any step to curb this kind of advertising. (By comparison, Twitter decided to completely ban political advertising on its platform.)

The primary reason that I found the Biden administration’s witch-hunt of Facebook distasteful was because there is an ongoing search for the convenient scapegoat.

While Trump was at the helm, Biden did not have to go looking for a scapegoat. Trump made himself readily available too, Trump thrived on anti-Trump rhetoric, while Biden’s supporters wanted to see him frustrated by Trump’s apparent “childishness” and self-centered approach to crises. They were in a mutually beneficial relationship. Trump says something crazy, Biden and other Democrats say things about him, Trump calls them names, … so on.

Facebook’s approach to misinformation has not changed dramatically between the Trump and Biden presidencies. But Facebook banning Trump from their platform was a dramatic change. In an instant, they evicted the existing scapegoat. The cage stood empty as the Biden administration struggled to give the first dose of the COVID-19 vaccine to 70% of the eligible population. This failure started a race to fill the cage with a scapegoat once again.

The “Misinformation Dozen”, 12 people who are responsible for 65% of all vaccine misinformation in the US, would have been the obvious choice. But looking at their combined base of followers, it is obvious why they did not make the cut. Even at the peak of their popularity, they had 15 million followers and 62 active accounts. With a population upwards of 300 million, I can understand why most people would be suspicious if the administration tried to blame a group with direct access to less than 5% of the total population².

I have one final criticism of the administration’s stance. As it has railed against Facebook, they have repeatedly asked for information from Facebook. But these requests seem to be non-binding. The administration, undoubtedly, has the ability to open investigations, subpoena documents and data directly from Facebook. Publishing this information is key to Biden’s argument about Facebook being responsible for the stalled vaccination campaign. But to do that, time-consuming work that (even) Biden’s party colleagues are probably unwilling to take up is a necessity.

The measured approach of compelling Facebook to share information, publicizing it and then, (finally) placing blame, requires work that lawmakers would forgo. Sensational comments are just as effective in the court of public opinion and can rile up supporters against the chosen scapegoat³.

This Media Matters report about the vaccine misinformation claims on Fox News is worth reading. Here’s the topline from the summary: “In the 2 weeks from June 28 2021 to July 11 2021, 57% of 129 segments about the COVID-19 vaccines had claims that undermined immunization efforts. 45% of segments included claims that the vaccination drive is coercive or that it represents government outreach. 37% of segments included claims that the vaccines are unnecessary or dangerous.” ↩
These 15 million followers are the tip of an iceberg whose real size is unknown. The followers are bound to be members of groups and probably share content from the Misinformation Dozen amplifying the content’s reach to several million more people. Facebook is (understandably) unwilling to share this information voluntarily. The Biden administration is (inexplicably) unwilling to force Facebook to share this information through legislative or judicial pressure. Blaming 12 people is simply not viable. ↩
I admit that on seeing a news article saying that the Biden administration was compelling Facebook to share data, I would not have written a post like this one. My guess is that Opinion desks would have written something but this reaction would not have generated half the news headlines that the sensational comments did. ↩

Monthly Recommendations (June 2021)

2021-06-30T00:00:00+00:00

This month’s theme is disorder, dishonesty and dysfunction. In that spirit, this month’s list includes the story of a missionary dabbling in things that she should probably have stayed far away from, the profile of a Trump administration operative who was an exceptionally bad manager and irritated nearly everyone whom he worked with, a list of the winners of the recent absurdity in the US stock markets and how it has helped those who got in early reap huge benefits by selling what basically amounts to snake-oil, and, a contemporary historical study of how hot-button topics on the Internet have evolved over the past 2 decades.

A Son’s Story (Zapana)

New Yorker (archived), 13th April, 2020

There is a terrible desensitization in the way that the newscasters switch from the number of people who died yesterday to how a team won a sporting match without missing a beat or giving the audience any time to recover from the previous story. A common argument for this is that people can’t be bogged down by every bad story that they hear on the news because then they will be unable to function; lest they be overwhelmed by the amount of bad things that are going on simultaneously around the world, the audience must be protected through this desensitization, some argue. This essay runs against the grain and tells the heart wrenching story of a death in the family.
How Stephen Miller Manipulates Trump to Further His Immigration Obsession (Blitzer)

New Yorker (archived), 2nd March, 2020

The former President of America, Donald Trump, had a “crazy clown posse” of henchmen and assistants. Some of these people were people like John Bolton who had an agenda and were working with Trump only because he was a tool to get that agenda done. Stephen Miller, on the other hand, was a new animal who could exist only in the world where his boss was someone who did not read briefings and allowed meetings to turn into shouting matches. The part about Miller wanting things to be done incredibly quickly, even if they were not done properly enough (legislatively) to withstand future legal challenges and to last long, was an interesting real-world example of why “move fast, break things” is a bad idea. His constant berating of the people that he met with seems like a managerial tactic to avoid.
The SPAC King Is Doing Just Fine Even as the Bubble Starts to Burst (Faux)

Bloomberg Businessweek (archived), 17th May, 2021

US stock markets have been crazy in the past few months. One of the other weird bubbles that has come up during that time has been the “Special Purpose Acquisition Company”, or “SPAC”, bubble. Levine explains what they are and how they work. This article is about the SPAC king “Chamath”, a megalomaniac, Elon Musk-type character who started appearing on everyone’s YouTube feeds and gained a lot of popularity in early 2021 when he started saying things like “Venture capitalists are conning everyone” and “Warren Buffet is not relevant, I am the new Buffet” and became a folk hero to the stonks traders. I am not sure how he got all this attention in a short period of time. He was one of the first few people to turn the SPAC concept into a bubble which would mint money for anyone who “got in early”. This article is about how he has come out on top, despite making deals with companies that have questionable viability and that almost everyone else is losing money on.
The Mission (Levy)

New Yorker (archived), 6th April, 2020

A fascinating recounting of the story of a woman who goes to Uganda and works as a missionary there. She works in the health sector and begins treating people. But then, children suffering from malnutrition start dying. Questions are asked about what she is doing and how she is so closely connected to so many deaths. Her intentions appear to be good and it is impossible to parse out what was really going on from this article. It is a good portrayal of the cutthroat nature of communities of do-gooders in lower-income economics that seem to have personal enmity and ego clashes in abundance.
- Date added to this list: <2021-05-30 Sun 11:27>
- Source: Newyorker 2020-04-13
- Author: Levy
- Note: Woman in Uganda who was accused of being negligent which lead to the death of several children in that country.
Online Culture Wars (Alexander)

Astral Codex Ten (archived), 11th May, 2020

The concept of a “culture war” has seeped into the mainstream now. There has been some discussion about how the conservative pundit leans on culture wars to generalize a single event and taint how a religious or ethnic group is seen by the majority. There has also been talk about how liberals wage culture wars against people by focusing on a wide variety of past oppression and inter-sectional identities, ensuring that everyone is uncertain about whether they are the oppressor or the oppressed in a given situation, and consequently, much to the liberals’ delight, everyone is walking on eggshells. Neither of those positions is completely true or completely false. It helps to look at the culture wars that have been waged on the Internet. Using some word frequency plots and his general knowledge of the Internet blogosphere’s history, Alexander shows that the extremely popular topics of a few years ago have been relegated to lame and uninteresting (the “I am so rich I dress in leisurewear when I go to a high-end restaurant” attitude).

More Access Leads to Better Governance

2021-06-28T00:00:00+00:00

Here’s the noble lie about democracy: The electorate holds the ruling party accountable for their actions during the next election. This is taught in classrooms and in political science courses; it is drilled into the minds of children who have the good fortune of growing up in a liberal democracy. The truth is revealed when they realize that the electorate is unwilling or unable to hold the ruling party accountable and that the ruling party is able to entice the electorate with populist schemes that are announced hours before campaigning begins¹. There is a tedious and disheartening conversation about why no country has realized this ideal state of democracy in practice or been able to stay in that state for a long period of time; a conversation that would identify the media, political self-interest and capitalism as a few of the causes. This post does not engage in that conversation. Instead, I focus on a core component of this system which would hold the noble lie up: Access to public services such as education, healthcare, and a competent local government. I contend that widening access is the silver bullet which will allow countries to reach the ideal state of democracy and maintain that state in the long term.

The Noble Lie

Let’s look at the lie in some more detail:

The electorate votes for a political party every few years
The elected political party forms a government and works for the welfare of a majority of the people
When the next election comes along, the political party campaigns on their credentials. Namely, the changes they brought about in society during their term in office and their plan for the future, as documented in their election manifesto
The electorate hears all this rhetoric and looks objectively at the government’s performance
At the voting booth, the electorate holds the government accountable for their actions during their past term, by voting for either the same party, signaling that they did a good job, or for the opposition party, signaling that they did not live up to the electorate’s expectations
If the ruling party loses, then they go back to the drawing board and await the next election
Whichever party lost the election will become the new opposition and will keep a close eye on what the government does. They will find counter-arguments to every policy that the ruling party puts forth, and this is an important part of the electorate’s opinion forming process
GOTO Step (1)

At least one of the above steps is broken in all of today’s liberal democracies².

Step (2) has never really worked. The government is focused on winning the next election and does what will work out best for their current and potential supporters; not bothering too much with the ardent supporters of the opposition party. The stickiness of political ideologies supports this approach. If one were to assume a utilitarian point-of-view, the limited resources that the government has access to must be used to safeguard their path to office in future election cycles. There have been a handful of astonishingly selfless politicians, such as those inside the conservative party in Australia who implemented gun control at the cost of their political futures following a mass shooting. These politicians are the exceptions that prove the rule.

Step (4) is universally broken. The electorate learns about what the government has done through the media and the media is regulated by the government. This is a conflict of interest for the government. Media organizations have their own ideal state which they have not achieved and the failure to live up to the requirements of step (4) are the result of knock-on effects of dependent systems, each of which is far from its ideal state. I have written about this problem before.

Step (5) is broken in countries where the government has expansive, unchecked powers over spending. In effect, the government is empowered to bribe the median voter through populist schemes. The government can thus side-step the requirement of “performing” when in office, but still convince voters on the campaign trail. There are also out-of-band methods³ like “Cash for votes” schemes to convince voters, though owing to their nature these are less frequently documented.

Step (6) is hard to adhere to for an enterprising political party. The ruling party is unwilling to reevaluate their policy platform or the changing demography of a country following an election loss. This refusal to change has lead to endless shouting matches on nightly news shows and walk-outs from legislative bodies that accomplish absolutely nothing in the Parliamentary system of governance that exists in India and Britain. It’s utility is questionable in separation-of-powers democracies like America. In an astonishing example from America, the losing side embarked on a national campaign to disenfranchise supporters of the winning side in early 2021.

Here, I will end this inventory of the failures of democracy and move on to my core argument.

What is “Access”? What are it’s Determinants?

“Access” is an ambiguous concept and risks becoming a catch-all term. I will define it concretely for our purposes here:

Access is the availability of public services like education, healthcare and a competent local government. These are things that private individuals can not do as efficiently as the government. Apart from these enumerated facets of access, we should consider pensions for the elderly, rations for the poor, opportunities for the unemployed and public transport for all as essential public services.

I think there are two determinants of how much access someone has during their formative years.

First, Access is heavily impacted by geographical location, with people in urban areas generally having better access due to the higher per-capita government revenue and the government’s enhanced ability to recruit talent willing to inhabit these areas. People in rural areas generally have little or no access to essential services and are often compelled to go considerable distances to procure them. Anecdotes about Indians walking several kilometers every day to go to school or college are commonplace even as recently as the 1990s or 2000s.

Second, Access is determined by socioeconomic position. The wealthy landowner in a rural area has the ability to shuttle their children to school or move closer to the school for a period of time, while everyone else is left behind. Further gradations become visible as children move to higher levels of education; the entrance exams for several Engineering colleges in India are held only in English and Hindi⁴, creating a major barrier for students who studied in schools where a regional language was the medium of teaching.

This difference within members of the electorate profoundly impacts what they do as participants in the democratic system.

Why is Access Important?

To understand the importance of access, I will break the election process into three separate stages: the pre-election stage of electoral polls and campaigning, the voting stage, and the post-election stage of measuring government performance.

Before the election: Polling and Campaigning

The impacts of access at this stage are under-discussed. Political parties use private and public polling⁵ to gauge what the electorate wants to hear them talk about. Who gets polled depends on how much access they have. The most common method for polling in America is calling people up on telephones. Someone who does not have a telephone is not likely to be polled and their opinion remains unheard by political parties before the election.

In smaller economies and middle-income countries, the problem becomes even bigger. Internet penetration in India is about 45% which reduces the utility of online polls like this one.

Looking at traditional polls like India Today’s Mood of the Nation 2020 poll, we notice the difference between the country’s literacy rate and the literacy rate of the people who were polled; while about 75% of the people in India were literate in 2018, about 96% of the people polled were literate. This gap makes the poll less representative because only 1 out of 5 illiterate people in India got the opportunity to voice their opinion. The poll’s coverage of the whole country is also spotty with only 97 of the 543 Lok Sabha constituencies surveyed. This poll is decidedly not the mood of the nation. Rather, it’s the mood of the people within the nation who have the most amount of access⁶.

Campaigning is less of a problem related to access because political parties campaign in places where there are persuadable voters. These places are generally chosen irrespective of the amount of access they have. A politician will travel as far as they need to if they can get an audience with a few thousand persuadable voters.

So, if you (or counterparts in your cohort) are not being polled routinely and diligently, then your options for getting your opinions to politicians are limited. One potent option which is widely recommended is to “get involved in local and regional politics” and talk to the politicians who are closest to you and explain your concerns to them. Each democracy provides it’s own method of accomplishing this.

America provides the public forum and town council meetings, whose inefficiency and ineffectiveness was immortalized by the excellent TV show, Parks and Recreation. Decentralized democracy generally provides multiple levels of politicians starting from a local counselor or panchayat leader (in towns and villages in India) going up to a member of the state or federal legislature. I think that this option is mostly touted for show. People at these levels are impossible to find, meet or talk to. Their functionaries are equally powerful and (generally) just as absent. Above all, this assumes that the duty of a citizen extends well beyond voting in an election: If they want to get their opinions out, they have to find someone who can effect change and proactively tell them about it. This seems like a dream and is not backed by anything that exists in reality. This ability to be proactive is again dictated by socioeconomic situation, with people who are well above the poverty line with more flexibility at their workplace to take time off and meet with the people who matter⁷.

There are some caveats to my opposition to this idea. The implementation of democracy has varying levels of imperfection across the world. Thus, theoretically, it is possible that a democracy somewhere in the world has really good processes which allow the voice of everyone to be heard. At scale, I don’t think that expecting citizens to be proactive is acceptable. Incentivising the government and political parties to employ private companies who do surveys and weight the results appropriately to represent the whole country sounds like a much better option to me.

During the election: Voting booth

Access to the voting booth is an important part of democracy that democracies generally do better at. There are few complaints about these. While citizens might struggle to get their voices heard before an election, when an election comes around, politicians would generally like to lock down the reliable voters who will vote for them and will be in favor of policies that bring them into the fold.

I will offer one minor and one major instance where access to the voting booth is systematically limited.

The minor caveat comes from India. India runs the world’s biggest election every 5 years. In 2019⁸, 910 million people were eligible to vote, while 610 million people actually voted, resulting in a polling percentage of 67%. These votes were cast over 5 weeks to elect 543 Members of Parliament. These numbers are staggering. The polling percentage in particular is comparable to American turnout during the 2020 Presidential election, an older democracy which has several features for convenient voting such as early voting, mail-in ballots, and same-day registration, which are missing from the Indian election process.

The lack of these convenience features makes it nearly impossible for someone who isn’t close to their assigned voting booth in an election. For anyone to vote in an Indian election, they must be registered to vote and have a voting slip. The voting slip specifies the polling booth that has been assigned and one is required to present oneself at the polling booth, on the assigned day, between 9 am and 5 pm (or a similar time period). This geographical restriction is an annoyance and creates a bias towards people who stay in the same place for a long time. In an economy that is still largely agrarian, these concerns are inconsequential⁹.

The major instance of systemic denial of access to the voting booth emanates from the politician’s desire to empower reliable voters who will vote for their party, while simultaneously finding dubious methods to disenfranchise reliable voters of the opposition party. There are two ways to deal with reliable opposition voters: persuade them that one’s party will product better outcomes for them or exclude them from the electorate.

There is really nothing more to say about this; if democracy is about convincing the electorate to vote for you and you are in charge of including and excluding people from the electorate, it makes complete sense to prevent the people who will not vote for you from being a part of the electorate. The Republican party in the American South decidedly went over the cliff on this in early 2021 and wholeheartedly committed to disenfranchise a large block of voters who turned out for the opposition in the previous election; their policies have been outrageous and they have not bothered to even present these policies in a palatable form. This kind of disenfranchisement was going on in the form of Gerrymandering, the American process of periodically redrawing districts to bunch up one’s own supporters into reliable constituencies, while breaking up the opposition’s supporters into multiple districts ensuring their inability to forge a plurality anywhere. While that was an obscure form of distorting the electorate’s powers, preventing Sunday voting is a direct and indefensible form of exclusion.

Similar legislative policies were nearing implementation and were widely protested in December 2019, before the COVID-19 pandemic forced the government to apply the brakes and focus on a separate crisis. The Indian government’s determination to implement the various citizenship tests and registers appears to be undaunted and with 3 years remaining in their current term, it seems entirely possible that they will implement at least a part of the agenda that they left unfinished before the next election in 2024.

After the election: Measuring government performance

Approval ratings are calculated using surveys that ask people some form of a straightforward question: “Do you approve of the current government’s performance?”. By keeping the question simple, they aim to capture the public’s approval of the job that a government is doing, implicitly assuming that the public will take into consideration all the various things happening in the country. It is a purposefully coarse metric which tells people about the present. It is not very helpful in the long run, except for direct comparisons between different leaders (even in comparison, the usage of approval numbers is suspect because of the differing conditions that a leader has to deal with when they come into office).

Morning Consult’s weekly poll gives us some idea of the approval ratings for leaders around the world.

The methodology of the survey determines the final number and different numbers can be found for America on FiveThirtyEight’s approval poll average and for Japan on NHK, the state broadcaster’s monthly survey of about 2000 people.

These numbers can be easily goosed and have wide short-term variations. For instance, when Modi, the Prime Minister of India, announced a nationwide lockdown to contain the COVID-19 pandemic in March 2020, the 7-day moving average of his net approval shot up from 52% on March 21st (3 days before the announcement) to 63% on March 27th (3 days after the announcement). This increase is not meaningful, as the effects of the lockdown were not visible in that short period. When the disastrous effects of the lockdown became apparent, the net approval returned to it’s original level around 53% in June 2020, nearly 3 months after the initial announcement.

Measuring the Government’s performance is a tricky task for any citizen. Access plays a pivotal role in this; specifically, the citizen’s access to information, their ability to process this information, and their ability to come to an independent conclusion (free of peer pressure and intimidation). All these facets of access are determined by the amenities available in her neighborhood in the present and the opportunities she was able to avail in childhood. People who are illiterate or physically challenged have been systemically excluded from the information ecosystem, although the advent of accessibility tools such as screen readers and platforms such as YouTube with content in several regional languages is gradually filling this access gap. We should be cautious in becoming too optimistic though as access to the Internet is far from perfect, even in some advanced economics.

Free and good quality primary and secondary education is also facing several obstacles and on this front, there is a wide gap between various countries. When one looks at the amount of time that schools and other educational institutions were asked to stop functioning during the COVID-19 pandemic, I think one can gauge a government’s attitude towards the education of their citizens. While schools remained closed in America and India for several months, Japan and France rushed to open their schools as soon as the first waves of the pandemic ended and doctors were able to understand the disease better.

As evidence mounted showing that the spread of the disease in schools is limited and can be curtailed using common-sense measures such as masks and social distancing, some countries responded by implementing these measures and reopening schools, while others decided to forget about them due to upcoming elections. The Indian government went one step further and canceled the year-end exams for CBSE, a nationwide education board, impacting 15 million students who were to take these exams in April 2021. The exams were canceled without any clues about what their replacements will look like and the whole thing was a huge communication failure which left a lot of students anxious about both the near- and long-term.

These decisions were made at different points during the pandemic and support for them is divided among the electorate. I don’t know of any poll that has asked people about their opinions on how important it is to open restaurants and movie theaters compared to opening schools. The Economic Survey in 2020 found¹⁰ the drop-out rate in secondary education in India (age 10 to 17) to be about 19.89%. This number is bound to have increased during the pandemic. It is a near certainty that the past year of school closures will increase social inequality in the coming decades.

Archived (2021-06-03) ↩
This is probably a pessimistic view, an overstatement and a simplification of the nuanced reality. But the inaccuracy of this statement is well within the tolerances that I have set myself for evaluating the performance of a democracy. ↩
Archived (2021-06-03) ↩
Archived (2021-06-03) ↩
For this post, I consider polling to be the systematic science of calling up probable voters and asking them questions, in order to gauge the public’s opinion on social, political and economic issues. ↩
I have made some assumptions here. Namely, I have assumed that this poll covers the easiest places to survey: urban and semi-urban places where access to telephony is high and people are reachable by various modes of communication. The names or economic attributes (such as the median income of the polled constituencies) of the 97 constituencies that were polled are not available in the “Methodology” section of the poll. Given this lack of information, I have decided to not give the benefit of the doubt to the polling agency, because they have not explained their weighting of the survey results or whether they have employed statistical processes to deal with minor inaccuracies in surveys or the differences between the sample’s population and the country’s population. ↩
Shanghai (2012), an Indian movie, is a good reference for learning more about how the bureaucracy works in India. The New York Times podcast series, Nice White Parents, is a good example of how a motivated group of people, with a lot of time and willingness on their hands, can bend the system’s rules to favor themselves and their group. A deeper discussion of both these works will not fit in this (already quite long) post, and I decided to drop them and focus on the core argument here. ↩
Highlight Statistics from the 2019 General Election in India ↩
In a hilarious turn of events, a successful movie in 2018 showed the protagonist, loosely based on Alphabet CEO Sundar Pichai, coming back to India to cast his vote and the drama that ensues when he realizes that someone else has illegally voted on his behalf. ↩
Statistical Appendix to the Economic Survey of India 2020-21. See p. 173. ↩

Notes and Review - Anna Karenina (Tolstoy)

2021-06-23T00:00:00+00:00

Anna Karenina is a love story. Reading this book has shown me the source of several scenes from iconic romantic movies: Lovers meeting in a railway station, rejection at dinners, and heartbroken characters traveling abroad attempting to heal their soul. Tolstoy’s descriptions of people and their interactions with each other were more compelling to me than his descriptions of natural scenes. This preference is probably related to my (current) personal preference for stories which have interesting characters doing unpredictable things.

Tolstoy has written beautiful characters in Anna Karenina. These characters are complex and relatable, despite having been written about 140 years ago. They are infuriating and lovable, and my attitude towards them changed as swiftly as the word they had spoken reached the listener’s ears. These characters have basic recognizable characteristics: love, fear, contempt, dreams, and anxieties. The description of Anna, Vronsky, and Levin is exquisite. Listening to them talk throughout the book, I have a good idea of what each of them would look like and how they would react in a new situation. Their descriptions are often through the eyes of their lovers and the inherent biases of this perception are a challenge to parse out (For example, I felt that the way people in society described Anna’s appearances was at odds with the way Vronsky describes Anna).

I noticed that Tolstoy gives the reader a very narrow window into the character’s true feelings. A character’s thought is framed in an extremely long sentence, which reveals the character’s true feelings near its culmination. The use of pronouns can often be confusing and can lead to confusion about whom (or what) the character is referring to when they say “them” (or “it”). This technique is used in a scene in which a professional artist, Mikhailov, visits Vronsky, a budding unskilled painter who is infatuated with art for the time-being. Here’s what Mikhailov has to say after looking at Vronsky’s painting:

One cannot forbid a person from making himself a big wax doll and kissing it. But if this person were to come with his doll, sit down in front of a man in love, and begin caressing his doll in the way a man in love caresses his beloved, the man in love would find it distasteful. Mikhailov experienced the same kind of unpleasant sensation when he saw Vronsky’s painting: he found it ridiculous, and annoying, and pathetic, and offensive.

What starts out ostensibly as an admission of the various forms of art, becomes a scathing criticism of people who engage in art as a hobby and ends as a dramatic take-down of Vronsky’s art. I can empathize with Mikhailov’s feelings of distaste while simultaneously feeling some sympathy towards Vronsky and respect for the effort he has put into his painting.

While Tolstoy applies this technique adeptly when characters express their displeasure with someone they are supposed to be in love with, he also uses it to show the duality of love. Once again, the description of love between two characters as described by the man in the relationship (I have removed the names of the characters to prevent spoilers about the story’s trajectory):

The preoccupation she had with small details, which was so at odds with his ideal of sublime happiness in the early days, was one of his disenchantments; but this endearing preoccupation, the point of which he could not understand but which he could not help loving was one of his new enchantments.

The preoccupation that this woman has with “small details” is a disenchantment, an enchantment, inexplicable and endearing to the man; all at once. One would be hard pressed to conclude anything about how the character really feels. Tolstoy has succeeded at keeping the reader guessing about how the character truly feels towards his lover (The reader suspects that the man is hopelessly in love with the woman and is trying to explain away some inconvenient details about her that he is unable to accept; it might also be true that the man is actually at the edge of his patience with the woman; there is no way to resolve this doubt for we see only a short period of the two characters’ lives together).

I read Rosamund Bartlett’s English translation of the book and there were several references to the characters switching effortlessly between French and Russian throughout the book. This book was written in the late 1800s and the dominance of English as the language of the elite classes was not yet cemented.

My advise for future readers of Anna Karenina

I have three pieces of advise:

Spend about 30-45 minutes on the introduction, familiarizing yourself with the way Russian names are structured and how characters switch between various forms of address depending on their intimacy with the person being addressed. Doing this helped me go through the rest of the book without having to think about how Stiva and Oblonsky are the same person and what it means for him to be referred to using either.
Keep a record of how you feel about a character after having known them for a while and track their evolution and how you feel about them throughout the book. It would be impossible for me to say that I love or hate any character; I can definitely say that I started out loving Karenin (Anna’s husband) and ended up finding his weak-mindedness disgusting. It’s very interesting to explore your feelings about this person and track the changes after having read the book and forming an opinion of them.
Use a video like this one to decide which translation to read. I don’t think it matters what translation you read as all of them are probably closer than one thinks and a first-time reader (like myself) could hardly have told the difference anyway. Watching a video and scoring the translations in a blind taste test on expression of images, simplicity of language, and overall score, helped me choose an edition and get on with the hard part: reading the novel.

Quotes

People who work for the common good

Konstantin Levin regarded his brother as a man of immense intellect and education, who was noble in the highest sense of that word, and endowed with an ability to work for the common good. But in the depths of his soul, the older he became, and the better he got to know his brother, the more frequently it occurred to him that this ability to work for the common good, which he felt he lacked totally, was perhaps not a virtue but, on the contrary, a deficiency of something—not a deficiency of good, honest, noble desires and tastes, but a deficiency of life force, the thing which is called heart, the impulse which drives a person to choose one of the innumerable life paths open to him and stick to it. The more he discovered about his brother, the more he noticed that Sergey Ivanovich, and many other people who worked for the common good, were not drawn with their hearts to this love of the common good, but had worked out in their minds that this was a good cause, and took it up for that reason alone. The observation that his brother did not take the question of the common good or the immortality of the soul any more to heart than he did a game of chess or the ingenious construction of a new machine only served to confirm Levin’s hypothesis.

Life and Death

Death, the inevitable end of everything, confronted him for the first time with irresistible force. And this death, which was there, in this beloved brother, groaning in his sleep and from force of habit indiscriminately invoking first God then the devil, was not nearly as remote as it had seemed to him before. It was in him too—he could feel it. If not today, then tomorrow, and if not tomorrow, then in thirty years—did it really matter when? But what this inevitable death was, he not only did not know and had never thought about, but also did not feel competent or bold enough to think about.

…

He sat hunched up on his bed in the darkness, with his arms around his knees, and thought, holding his breath from all the concentration. But the more he concentrated, the more apparent it became to him that this was unmistakably the case, that he really had forgotten, overlooked, one small factor in life—that death would come along and everything would end, that it was not worth even starting anything, and there was no way this could be helped. Yes, it was awful, but that is how it was. ‘But I am still alive. So what should I do now, what exactly should I do?’ he said with desperation.

Marriage

She was both appalled by this indifference and thrilled by what had caused it. She could not conceive of or wish for anything beyond life with this man; but this new life did not yet exist, and she could not even picture it clearly in her mind. There was only anticipation—fear and joy of the new and unknown. And now, in just a few moments, the anticipation, the unknown, and the remorse at renouncing her old life was about to come to an end, and something new would begin. This new thing could not but be frightening as it was unknown; but frightening or not, it had already taken place six weeks earlier in her soul; today they were merely sanctifying what had happened a long time ago in her soul.

When, as a bachelor he used to observe other people’s married life, their petty worries, quarrels, and jealousy, he would just smile contemptuously in his soul. He was convinced that not only could there be nothing of that sort in his future married life, but he also believed that it would be completely and utterly unlike the life of other people even in all its outward forms. And instead of that, he suddenly found that not only had his life with his wife not formed into something special, but, on the contrary, was entirely formed of those same trifling details which he had so despised before, but which were now against his will acquiring an extraordinary and unassailable significance.

Jealousy

And jealousy of him led her to resenting him [her lover] and seeking grounds for her resentment in everything. She blamed him for everything that was difficult about her situation. The agonizing state of suspense between heaven and earth which she had to endure in Moscow, her husband’s slowness and indecision, her isolation—she put it all down to him [her lover]. If he [her lover] had loved her, he would have understood all the difficulty of her situation and extricated her from it. And he was the one to blame for the fact that she was living in Moscow and not in the country. He could not live buried in the country as she would have liked. He needed society, and he had put her in this dreadful situation, the difficulty of which he did not want to understand. And he was likewise to blame for the fact that she was forever separated from her son.

War

Alphonse Karr* expressed it very well before the war with Prussia: “You consider war is necessary? Excellent. Whoever preaches war should join a special front-line legion and lead the assault into the attack in front of everyone else!”

Beware, Spoilers Ahead

The following two sections of the review contain spoilers. These spoilers will reduce the suspense in some parts of the book and reveal the trajectory of the story (such as who ends up with whom). If you plan to read Anna Karenina in the future (and I recommend that you do!), I would advise you to not read any further and dive straight into the novel instead.

Iconic Scenes

The railway station scene in which Vronsky meets Anna as she emerges from the train and he is entering the compartment, is perhaps the most iconic scene. There are other scenes which I found touching. I liked each of the following scenes for their ability to build suspense in the reader’s mind and hold the reader’s attention by conveying the raw emotion felt by the characters involved.

Anna and Vronsky at the derby: Vronsky’s visit to the place where Anna is staying at to declare his love for her; Anna’s disregard for Karenin’s presence at the derby; her unveiled anxiety for Vronsky and her hope of his victory at the horse race. This scene’s suspense builds to a stunning climax. My heart was inexplicably racing and I wanted to quickly absorb as much of the scene as possible and get to the conclusion and see how each of the characters fared
Anna’s reunion with her son Seryozha: Anna returns to her home to meet her son Seryozha in the morning. She is let in by the maids and housekeepers who work in the house and she goes to his room and waits for him to wake up. When he finally does, they have the most touching reunion that I have ever read. This scene was a perfect expression of the tender love that exists between a parent and a child. Anna’s parting words to Seryozha are packed with pain and emotion. The writing is amazingly articulate. Anna implores Seryozha to love his father and she says to him that she has wronged his father. But it is also easy to perceive that she knows that Seryozha will never love his father as much as she loves her. Seryozha’s father, Karenin’s attitude towards his son is a stark contrast to the emotion that Anna shows in this scene
Levin’s marriage: Levin’s marriage was another particularly good section of the book. Levin’s marriage to Kitty is described over multiple chapters and the things that Levin says during the preparation for his marriage and on the day of his marriage feel extremely normal and close to reality. In particular, on the day of his marriage, Levin is in the hilarious predicament of not having a new shirt to wear to his wedding. The procurement of this shirt takes an unreasonably long time as his butler has to unpack his clothes suitcases and get the shirt back to where he is. During this short period of time, Levin, Kitty and everyone else impatiently waiting for the wedding to begin at a church feel the general anxiety that accompanies big events in someone’s life. This description of anxiety is helpful and I will return to these sections of the book in the future
Levin throwing Vesolvsky out of his house: Veslovsky is an old friend of Levin’s wife, Kitty. He arrives in their house for a hunting trip and stays a couple days during which he shamelessly flirts with Kitty. This irks Levin no end, and eventually he decides to throw Vesolvsky out of his house. Having been completely irritated by Vesolvsky’s antics up until that point, seeing him be thrown out of the house was a lot of fun and very rewarding (considering the anxiety I felt about Levin and Kitty’s relationship after this fool had meddled in it).
Levin and Kitty at Levin’s brother Nikolay’s death bed: Levin visits Nikolay’s death bed after getting a telegram from Nikolay’s lover telling him that his brother is sick. After some deliberation, Kitty convinces Levin that she must accompany him to his brother’s deathbed. Levin’s initial frustration with Kitty’s insistence and his perception of this insistence as selfishness in not wanting to be left alone is perplexing. When the two of them arrive at the location where Nikolay is staying, Kitty is the exact opposite of Levin: She takes control of things and does things to make Nikolay more comfortable; while Levin is paralyzed by the state that he finds his brother in and is unable to think of what to say or do. Levin admires Kitty’s presence of mind as she goes about talking endlessly with Nikolay and making him comfortable in the final few days of his life by cleaning the room and improving the environment that his brother lay in. I have felt awkward and out-of-place (much like Levin) in similar situations and admired people (like Kitty) in my life who are not afraid of death and can do the things that are required, without regard for the long-term philosophical consequences which Levin and I use to explain our inaction and paralysis. This scene showed me the ideal to aspire to as a compassionate friend or relative
1. A coworker pointed out to me that Levin’s attitude towards Kitty in this section, when he says things like “While Kitty can’t say half as much about death as me, she understood it and was not afraid of it”, was condescending. I found this attitude to be a defense mechanism that Levin was using to justify his ineptitude in the face of a distressing situation. But I can’t find a good argument against my coworker’s conception of the scene and I feel obliged to put it here
2. The same coworker also pointed out to me that the repeated talk of “masculine independence” whenever the female characters in the book express their desire to spend time with their husbands shows that Tolstoy was not more progressive than the average person of the era that he lived in

Characters

Throughout the book, Levin was the character I identified with the most. He is similar in age and temperament to me. He seems to be based on Tolstoy with some of Tolstoy’s hobbies attributed to Levin towards the end of the book (Tolstoy’s first name is “Lev”). While this similarity in age is important to me, as I get older, I wonder whether I will continue to related with Levin but simply think of him as my “past self” rather than as my “present self”, as I did on this reading.

Two of the three main male characters in the book, Karenin and Levin, have similar childhoods; they were both orphans with ambition, who did not have a lot of friends and had a craving for independence and the confidence that comes from having earned everything they possessed. Karenin’s “glacial confidence” when contrasted with the Vronsky’s nervous youthful energy was a confrontation I would liked to have seen more of (They meet only a handful of times).

Oblonsky is another character who is, in turn, hilarious, annoying and infuriating. At one point, when Oblonsky and Levin are out hunting, Oblonsky advises Levin to safeguard his “masculine independence” after he sees Levin writing back to Kitty and feeling anxious about a short 2-day separation from Kitty. He (I believe rightly) points out that Levin’s anxiety over such short periods of separation is unsustainable. I don’t know if what Oblonsky says is true for I have not had a similar experience to him or Levin. I can only speculate here: Levin’s anxiety felt normal and acceptable. Unsurprisingly, Oblonsky gives similar advice to Vronsky. Oblonsky is the essential link between Karenin, Vronsky and Levin, who rarely meet throughout the book, despite having the symbolic connection of being the male protagonists of the book.

For Karenin, Anna’s husband, I felt a dull disgust and repulsion towards the end of the book. I started out liking Karenin and even feeling sorry for him; but after Oblonsky’s meeting with Karenin, the Countess and a clairvoyant in St. Petersburg, I could only see how weak minded he really was¹. His conduct towards his son Seryozha does not inspire, while Seryozha’s confusion at his only remaining parent’s detachment is heart breaking.

For Vronsky, I felt contempt at his apparent entitlement and incredulity at his shifting nature. His love for Anna is human for he feels it strongly but still wants some distance for his own pursuits. Anna’s love for Vronsky is never ending and is defined by the reality that Vronsky has all the power in their relationship, both in her mind and in society’s opinion of their transgression; when they return to St. Petersburg, Vronsky goes out to the theater but implores Anna to not accompany him for fear of her “being disgraced”. Anna goes nevertheless and what Vronsky feared ends up happening. This section was not in vain because Vronsky comes to the stunning realization that the doors of elite society have closed for Anna and there was precious little he could do; I felt sympathy for this Vronsky. Right after, Vronsky shows that, as cavalier he might portray himself to his friends, he needs this elite society in his daily life. And for that Vronsky, I felt only pity, for he knew not what he had, in a lover as devoted as Anna.

And finally, for Anna, I felt a mix of love, hate, and pity. It would be accurate to say that I felt every possible emotion towards her character because she changes significantly during each phase of her life depicted in the book. Her initial introduction and her time with Kitty is charming; her time outside Russia paints her as impatient and needy; her time back in Russia enhances her inability to understand the realities of her new position in society. The way she lashes out at Vronsky and her friend Princess Betsy shows the petty parts of her nature. Eventually, she becomes paranoid and obsesses over unanswered notes that she knows have not yet reached her lover, Vronsky. Anna ends up terribly, terribly alone. Consequently, Vronsky stares down the barrel of a similar fate.

My notes from when I was reading the book are rather harsh towards Karenin. After the Oblonsky meeting: “That he acted the way he did in front of Oblonsky was a fucking disgrace - You are pathetic, Karenin”. Rather harsh. ↩

Monthly Recommendations (May 2021)

2021-05-31T00:00:00+00:00

This month, we deal with the recent past. The recommendations include the profile of a digital marketer, which looks back at how Trump’s 2016 campaign used the tools at their disposal effectively to achieve something the mainstream strongly believed was impossible, an article about the disastrous effects of healthcare systems being optimized for efficiency and the steps we can take to avoid them, and how loneliness might be one of those truly new and modern problems which time travelers would be unable to explain to their relatively temporally close counterparts in the late 1700s.

The Man Behind Trump’s Facebook Juggernaut (Marantz)

New Yorker (archived), March 9, 2020

This is a profile of Brad Parscale, a digital marketer with no political experience, who worked on Trump’s 2016 campaign and used Facebook effectively to do things that neither Facebook nor the mainstream thought were useful, possible or impactful enough to decide elections. He proved them wrong by diligently looking for the things that were possible within Facebook’s tools for Advertisers and trying everything. Noteworthy among the strategies that he employed was one which required campaign event attendees to enter their phone numbers to get free tickets. This data was then uploaded to Facebook as a spreadsheet and Facebook used the data in their databases to create a list of users who were similar to campaign event attendees.
What The Coronavirus Crisis Reveals About American Medicine (Mukherjee)

New Yorker (archived), May 4, 2020

Efficient systems are great: They are the holy grail of capitalism. The inefficiently used resource is detrimental to your bottom line. Healthcare is not a capitalistic endeavor though. So, when hospitals, schools, nursing homes, and kindergartens are run using these principles of least resource wastage, they are less resilient to major crises. This article is not a rant; It looks forward in a useful way. It is not just a survey of the problems that were in the system which lead to the disastrous outcomes in America last year; there are a lot of suggestions about what we should do. This article is focused on the American healthcare system. The Indian healthcare system is no different: Private players make up most of the market and insurance coverage is very low (in this regard, the Indian system is in a worst state than America, where coverage has expanded in the past decade). As India’s insurance market grows and more people get insurance, hospitals and the government have a good opportunity to side-step bad outcomes¹ in the future.
The History of Loneliness (Lepore)

New Yorker (archived), April 6, 2020

During the feudal era, loneliness was unheard of. During the Victorian era, even the wealthiest people had no privacy and were surrounded by housemaids and cooks who hovered around incessantly². Lepore argues that there has been a paradigm shift in society.

Before modern times, very few human beings lived alone. Slowly, beginning not much more than a century ago, that changed. In the United States, more than one in four people now lives alone; in some parts of the country, especially big cities, that percentage is much higher. You can live alone without being lonely, and you can be lonely without living alone, but the two are closely tied together, which makes lockdowns, sheltering in place, that much harder to bear.

I am suspicious of phrases like “paradigm shift”. Lepore does an incredible job at putting forth a convincing argument though. Measurable loneliness has increased in the past few years and this article gives the historical context which was missing from Vox’ excellent video on the topic, Why Are We So Lonely?³.
The Untold Story of How Jeff Bezos Beat the Tabloids (Stone)

Bloomberg Businessweek (archived), May 5, 2021

Say you own a leading newspaper and are famous and powerful enough for people to believe you and go out on a limb to do favors for you. Further, say that you are hit with a scandal that threatens to take away some of the wind from your sail and reduce you to a mere human. What do you do? Jeff Bezos has written the playbook on one of the options you have: Go on the offensive and relentlessly hound the tabloids that are trying to take you down using your media connections. This recounting of what he did and how he succeeded is bizarre, unexpected and reads like a crime-thriller novel. There are a lot of names and keeping track of all the characters gets to be a chore though.
Two Big Things You Need to Understand About Inflation (Smith)

Bloomberg (archived), May 11, 2021

Inflationary pressures are on the rise⁴. Central bankers are convinced that this is a temporary spike and that it will subside. There is a lot of economics jargon thrown around on publications that are focused towards savvy readers and I don’t understand any of that; so articles like this one clarify the situation for me, without dumbing down the issue enough to make it boring or “obvious”. Laypeople are often disconnected from economic news and seem not to care much about inflation unless it reaches a record-breaking number and newspapers start running front-page articles about it (like during India’s double digit inflation period in 2013-2014). But inflation plays an important role in daily life (should I buy 10 kg of rice before the RBI’s Monetary Policy Committee releases the minutes of their meeting this Friday?) and for longer-term investment planning (If inflation is at 5.5% and deposit interest rate is 4.4%, what term length should I choose for my fixed deposit?).

“I see fleeting moments of happiness in between extended periods of boredom and stress.”

New Yorker

This Newslaundry video has a good interview of what happened and the workarounds that were used by ingenious doctors and nurses to prolong their Oxygen supply. Unfortunately, the camera work is inexplicably shaky. The doctor says something that is disturbing for any healthcare consumer: “I am sitting here with narrow coronaries … I don’t know if I am supposed to be concentrating on treatment of patients or trying to get Oxygen [using Twitter and talking to officials]”. ↩
A reference from the TV show Downton Abbey and novels like The Remains of the Day (Ishiguro). ↩
If you want to spend more time listening to someone talk about loneliness dispassionately, then I recommend Marilynne Robinson’s interview on the Ezra Klein Show. ↩
See paragraph (4.) in the press release. Cost-Price Index has gone up in all places between March 2021 and April 2021. ↩

The Midnight Assemblies

2021-05-13T00:00:00+00:00

“The night is dark and full of terrors” - Parents use phrases like these to put children to sleep and to curb their adorable natures and their inexhaustible reserves of energy, which enable them to run around, squealing in delight, until all hours of the night. On exiting childhood, most people discover the deception that their parents had engaged in. The second years living in the Nehru Hall at IIT Kharagpur had a remarkable experience: For them, the terrors returned and their parents were proved right. They were woken up at odd hours of the night, requested to demonstrate the skills they had learned in the common room and the dining hall and subject to a series of deranged meditations. These polite requests were unlike those that some might be accustomed to, as they were made by third-, fourth- and fifth-years who stood in the first-floor corridor and directed their voices downwards towards throngs of second-years who stood in neat rows on the playground. This was the final form that hazing at IIT Kharagpur, affectionately referred to by the euphemism, “Orientation Program”, or simply, O.P, assumed. I recount my experience of O.P. in this 4-part series. This is the fourth and final part, The Midnight Assemblies.

On a few days in September and October each year, a group of third years would go to the top floor of Nehru Hall at midnight and begin waking up all the second-years who lived there. An onlooker might ridicule these amateur explorers; why! Some might even go so far as to call them malevolent. Allow me to dispel that notion: They were not malevolent; in fact, their intentions were pure. They aimed to better the lives of the second years who had had the good fortune of being assigned to this particular hostel through education, education that the institution can’t (or won’t) provide.

A part of the inscription on the Statue of Liberty says that tired, poor and huddled masses yearn to breathe free. What about the masses who are already free? How can one awaken this yearning within them? These third-years had the answer: turn the free masses into tired, sleep-deprived and resentful masses, so that they might rediscover their freedom. Philosophers have oft-repeated that losing what you believe you are entitled to is the only way to understand it’s true value. Today, I go to sleep knowing that not a single deranged person out there can awaken me to serve a strange power trip. That expectation, which is taken for granted by most, was a luxury when I was in college.

On a night like this one, an unbiased observer would witness several third-year students walking with impunity through a floor full of sleeping second-years. The observer would see them banging loudly on doors and peering shamelessly through cracks in windows. The observer would notice them relishing their power and reveling in the authority that had been bestowed upon them by Nature¹. They would be seen employing such innocuous strategies as entering uninvited into a room whose door had been left ajar, turning on the bright light or tapping sleepers on their shoulder in a not-totally-pleasant manner.

While the third-years performed their part in this exercise, they did not bother to hide their activities or “keep it down”. They feared no retribution. Unlike other hostels in India, the warden and the hall managers did not actually sleep in the hostel. They were probably asleep in their homes, dreaming about spending as little time as possible inside the hostel and ensuring that all of it was during the work-day². The only remaining adults in the hostel at midnight were the security guards, tasked with keeping bad people out of the hostel. They did not have any responsibility for the activities of the people who were already in the hostel though, no matter how morally corrupt they were or could be.

The hostel’s security guards were curious adults nevertheless. As soon as they realized that what happened every year was happening this year as well, they would scurry around and make themselves scarce. This was due to a welling up of noble self-interest. “Do not meddle in that whose conclusion is unknown to you” were the words they repeated in prayer each night. No one knew how the night was going to turn out or where the activities might lead this year. So, it would be most unfortunate if one were not to have access to the vital weapon in the coward’s arsenal: Plausible deniability.

A strain of arbitrariness runs through this narrative: Why wake people up at midnight? Why not before? Why wake them up at all? What could be gotten from this deprivation of one’s own sleep and that of strangers who clearly wished to remain uneducated? The first 2 questions can be answered satisfactorily. While throughout the day, second-years went in and out of the hostel as they pleased, everyone returned to their rooms at night. Being used to sleeping in their own bed undisturbed, after a day full of activity, the second-years had underestimated the diligence of their preachers and tormentors, who would gladly give up a night of sleep, if it meant that they could get drunk on power.

After waking the second-years up, they would be lead down to the playground and asked to stand in neat rows. Once this part was complete, the third-years went about summoning the bishops who would deliver the day’s sermon. These consisted of some fourth- and fifth-years. The people who held the titles of “Hall President (HP)” and “Hall Vice President (VP)” were invariably present and jointly played the role of the Master of Ceremonies. These titles were lofty and meaningless to the real world, as the letters of the alphabet are meaningless to an infant. Nevertheless, the play-acting continued. An infant immensely enjoys playing with a building block in the shape of the letter A and fully grasps the meaning and purpose of that building block: gratification of her desire to have a good time. And so the person who called himself the “Hall President” believed that there was meaning in the title as it served his purpose: the realization of their fantasy of wielding power over people who would never voice their opinion and inconvenience him.

The first things that the bishops noticed were the minor misalignments in the assembly: someone who was not neatly lined up, someone else who was sleepy. They took pains to point the person out and put them in their place by extolling on the virtues of discipline. In this regard, they behaved like a drill sergeant who had said something incredibly stupid and was being teased by his subordinates. Feeling anger rise up in his throat, the drill sergeant is harsh with his subordinates for the next few days. It’s not a proportional revenge; it is the only thing he knows how to do. If the thought that he should try and not make a fool of himself in the future had entered his mind at all, he did not give it away.

After the assembly was perfect and everyone was in line, the bishops began their sermon. I am certain the reader will excuse me for not remembering the content due to the exceedingly drowsy state I found myself in during these occasions. As these were fifth-year students who had the time and willingness to preach from a first-floor balcony at 1 a.m., one can safely assume that the content was primarily non-academic. My faint memory does contain a few choice nuggets of knowledge, akin to the crude teachings of 20-minute TEDx talks. Admittedly, they will be of no value to competent adults who managed to stumble into a workplace with respectful coworkers.

I remember one astonishing detail clearly. A few of the sermons were blessed by the presence of an alumnus who used to live in this hostel: An adult who had returned from the real-world and was now standing at a balcony at 1 a.m., discoursing at 100s of second-years whom he felt an inexplicable connection with, despite being a complete stranger to them. “It doesn’t make any sense!”, some of you might be exclaiming. Well, I can only say that your passion for your living quarters did not match that of those who had returned back to IIT Kharagpur and were reliving their glorious college days, possibly wresting back the power they had held once-upon-a-time.

The sermon regularly lasted anywhere between an hour and 3 hours. It did eventually end, but the party was not over yet.

After this late-night / early-morning sermon, one of the rituals was a “Tempo Shout”. It involved the assembled sleep-deprived second-years getting together in a circle, getting ready to sing kumbaya and shouting things. With hoarse voices, a lot of resentment, little energy, and even less respect for the seniors gazing upon them wistfully from the first floor, these students shouted the things that they were told to shout. Even those participants who were determined to deal with everything that was happening as a mere joke, which they alone were in on, gave up and participated wholeheartedly.

Another post-sermon activity can be explained only through psychology, due to its puzzling nature and the warm reception it received among members of the congregation. After the sermon, the tormentors and the tormented walked to a dilapidated food stall at the entrance of the college arm-in-arm. This stall served food rich in carbohydrates (namely, Maggi). Seniors showed their magnanimity by buying these plates of food for the second years who had been up all night and could look forward to a wretched, sleep-deprived, confusing day, which was made better only by the knowledge that they had learned important life lessons which remained hidden from their well-rested friends who lived in other hostels.

With this, the priming process that began in the common room several weeks ago was finally complete. In 2014, O.P. at Nehru Hall was a resounding success.

Programming note: This is the fourth and final part of a 4-part series.

Post	Date
The Curious Case of the Dweller Who Wouldn’t Move Out	10th May, 2021
The Sermon in the Common Room	11th May, 2021
Victorian-era Dining Hall Etiquette	12th May, 2021
The Midnight Assemblies	13th May, 2021

For they were born 1 year before the second-years now being rudely awakened from their slumber. ↩
The work-day for hostel workers in IIT Kharagpur began at 3 pm and ended at 5 pm. ↩

Victorian-era Dining Hall Etiquette

2021-05-12T00:00:00+00:00

Food holds a special importance in everyday life. A lot of memories are connected to it and apparently the most important word in French is the word for the afternoon snack. In IIT Kharagpur’s Nehru Hall, there were no such pretensions about food; but a strange reverence was shown to the place where food was often eaten, the mess hall. The single most important phrase in this hallowed ground was “Excuse Me”, the over-used English phrase most often heard in Victorian dramas when heroines flustered by their love interest utter these words and an escape hatch is opened to them. Second-years were implored: when you are done eating your food and are stepping away from the table, you must cast this magical incantation. Once cast, the spell worked wonders as it suppressed the bout of shouting and mocking from third-, fourth- and fifth-years. Curiously, everyone in the third-year and above inside the same dining hall was exempt from this rule. As in Pride and Prejudice, another Victorian-era drama, the female characters who are being married off have little say in whom they are married to, while the men have all the choice that they could possibly want; so in Nehru Hall, second years had to follow a lot of rules designed to keep them “humble and respectful”, while everyone else lead rule-free lives. This dichotomy was a part of hazing at IIT Kharagpur, affectionately referred to by the euphemism, “Orientation Program”, or simply, O.P. I recount my experience of O.P. in this 4-part series. This is the third part, Victorian-era Dining Hall Etiquette.

Closely following the sermon on self-introduction, everyone in the hostel appeared to be dead-set on driving arguably outdated customs and practices into the brains of second years. As their location, they chose the one place that second-years were bound to visit a few times each day: the dining hall (or mess hall or just, “mess”). The food that was served here was edible; I hesitate to add any other epithet. While eating this food required a profound acceptance of the status quo, the experience of second-year students in particular was improved by third-, fourth- and fifth-years engaging the hungry masses in spirited conversations about rules, the importance of rules, and adhering to rules. They also provided comic relief, by making jokes at the second-years’ expense. Laughing at a joke someone else makes at your expense is the age-old path to stress-relief.

The topics of discussion ranged from the slightly uncomfortable¹ to the deeply distressing². And like the proverbial lunch lady in kindergarten, the tormentors pointed out the slightest violation in etiquette among the victims.

Did you leave the table without excusing yourself? This would earn you an inexplicably long lecture about values, respect for elders and propriety. The lecture did run the risk of provoking a victim into an impassioned speech about the propriety of tormenting ostensible strangers at the dinner table. The victim might ask the third-years “What gives you the right?” I did not witness any such altercation though, and it was probably because all second years held these people in high regard.

Did you step away from the table to get food at the buffet but did not put your chair in a position indicating that it was occupied? This minor transgression of a deranged directive would be the trigger for an unsolicited meditation on the meaning of “rules” and the importance of following them when you are in your second year and disregarding them when you are in your third year³. Apparently, rules should be followed at least for a year, so that one gets a taste for what rule-following feels like. After that, the former rule-followers are promoted to be “enforcers”. I live by this even today and never follow any rule for more than a year; it has served me well in the real world.

Did you have the audacity to walk into the mess-hall in attire that exposed your bare knees? Well, this was the cardinal sin and would provoke a collective gasp in the form of a loud, screeching, and wholly unpleasant noise emanating from the exalted fourth-years. You would be gently reminded that the rules prohibited this kind of attire. Some second-years might be dispatched to the back of the dining hall, to read the list of rules and give the fourth years a summary of what was written there. Others might face the less loaded question, “Can you even read English?”. To this question, I present the answer that I prepared but never had a chance to deliver: “Yes, I can read English. Also, I am particularly glad that I can write English; for I intend to return to my room and write down what just transpired in a note for reference in the future.”

These conversations might appear normal or even friendly. Some might counsel the victim, “This kind of rhetorical questioning is how barbarians have broken ice for centuries. Cheer up!”. Yet others might console, “This was nothing compared to what happened in the dining halls in my college hostel …”. These people are right! I find the latter piece of nostalgic recounting particularly soothing. The world is famously resistant to change. What happened to your parents in college must happen to you in college. It will also happen to your offspring. Why else would the most memorable phrase about change be “the lack of change is the only constant”?

Some second-years managed to avoid the tables where these vultures scavenged for their almost-dead prey. Others employed devious methods and avoided these conversations altogether during the prolonged period of a month. These conversations in the mess could jump out of the dark and assault anyone, those who were complacent and had dropped their guard were particularly susceptible. Everyone knew that this was going on in the dining hall as they heard stories from second-year students who praised their seniors in curiously crude language. The anxiety felt by the about-to-be-hazed is identical in scale to the exhilaration felt by the already-hazed. At least the already-hazed can rest easy knowing that they have gained access to knowledge that is refused to people who have the misfortune of living in civil society!

To the diligent conductors of O.P., one second-year student left uneducated equaled defeat; “No man left uneducated”, was their creed. So, they took the fight to educate to the room in which these second-years slept after a long day. In the final installment of this 4-part series, I will relate the curious nights during which second-years stood in neat rows on the ground in the middle of the night, while fifth-years and even some alumni discoursed at them from the first-floor balcony.

Programming note: This is the third part of a 4-part series.

Post	Date
The Curious Case of the Dweller Who Wouldn’t Move Out	10th May, 2021
The Sermon in the Common Room	11th May, 2021
Victorian-era Dining Hall Etiquette	12th May, 2021
The Midnight Assemblies	13th May, 2021

“I don’t care about this at all” ↩
“What is he talking about and when will he stop?” ↩
A second-year student not following the rule book and a third-year student following the rule book were both subject to ridicule. ↩

The Sermon in the Common Room

2021-05-11T00:00:00+00:00

As a second year student at IIT Kharagpur, the 2.5 months between July and September 2014 were uneventful at Nehru Hall, one of several hostels inside the campus. Then, things took a turn. While I believe that it was for the better, the voice at the back of my head believes it was for the worse. In this post, I relate the story of a sermon that was delivered in the common room of my hostel. The sermon in question was about the art of introducing oneself to a group of strangers. While many might naively believe that they have mastered this art, this sermon will open your eyes (as it did mine) to the right manner of making a first impression. This was an integral part of hazing at IIT Kharagpur, affectionately referred to by the euphemism, “Orientation Program”, or simply, O.P. I recount my experience in this 4-part series. This is the second part, The Sermon in the Common Room.

Before the sermon, comes the priming. The preacher must establish his preeminence, ease the congregation into the environment where the sermon will be given and familiarize the congregation with the tone that will be used. In 1973 in an undisclosed location, a process similar to this priming is said to have taken 6 days. In that case, the pressure was higher and the stakes hardly comparable, and the results left experts baffled at this technique’s effectiveness in endearing the preacher to the congregation. In Kharagpur in 2014, it took about 10 days and achieved similar results.

On the first day, all the second-year students were courteously invited to the hostel’s common room, at 9 pm right after dinner. To the uninitiated, this sounded like a friendly invitation to get to know the seniors who were living in the same hostel: everyone studying in 3rd year and above. The assembly was moderately successful and about 100 people sat on the floor in a well-ventilated room. A group of about 15 third-year students introduced themselves in-turn. These students were central to the organization and smooth execution of O.P. Nothing in their introductions stood out and I shudder to even imagine my inattentiveness during this part. After the introductions, there were some “general” announcements and calls for auditions for various hostel teams: athletics, team sports, dumb charades and so on.

These calls to auditions were courteous: “If you can swim and want to be a part of the hostel’s swimming team, then talk to the captain of that team”. Then, the captain would take the stage and make a pitch to get people to come to the auditions or try-outs for their team. All the captains succeeded at making participation in hall activities sound like noble pursuits. These teams would compete in a yearly extravaganza known as the General Championships (or G.Cs) in which hostels competed against each other in a variety of competitions; points were awarded to the victor and the championship was awarded to the hostel with the most points. Every hostel had won this championship at some point in the past. Nehru Hall was no exception. But the captains were skilled pitchmen and made winning the GC sound like an exceptional achievement that no one else was capable or worthy of. They stressed that no other mere “hostel” could win the GC, because of the sheer desire that thrived exclusively at Nehru “hall”.

A rational member of the congregation might be expected to ask, “Well, how come you didn’t win the GC last year? Or the year before that? Or once in the past 5 years?”. I have never seen anyone asking questions at a sermon though, so I believe such a question would have been highly inappropriate. If asked, the question would have provoked a lively discussion about the reason that the GC was not won by our “hall”: Collusion between the external judges of an event and the people living in other hostels. These theories covered a lot of ground and the preachers did not balk at hurling allegations far-and-wide: “The judge was another hostel’s hall president’s nephew’s mother’s colleague’s son’s Guitar teacher! That was clearly the reason for the blatant bias shown by the judge during the event last year”.

These theories also included the Secretaries of each event, student representatives who were tasked with conducting the events and whose job it was to find judges for each event. These Secretaries were elected through an election where students voted and they were often from other hostels. The mere fact that the people who were choosing the judges were from a “hostel” was proof for many that there was a massive conspiracy against our “hall”.

On listening to this answer, the rational member who had asked the question would fall silent and introspect quietly at the lack of wisdom which had caused them to begin this discussion. Everyone else thought about the theories just put out and continued to listen with rapt attention to the preachers, who were talking about fantastical things that had happened in the past which had lead to the crushing defeat of our hall in the G.C.

The number of spectators during these post-dinner “group hangs” in the common room remained steady with mostly repeat participants. I was fascinated by the determined group of preachers who had taken it upon themselves to educate the second-year students. What other proof could I provide of their magnanimity to you, dear reader? This priming period worked phenomenally well as the third-years conducting these sessions quickly rose to prominence in second-year circles and were revered as people whom one could look up to for advice in a tough situation.

On the 11th day, things changed dramatically. While there was no outward change in the structure of the session, there was more shouting; much, much more shouting. The topic of this session and several sessions after this day was deceptively simple: Introducing oneself to a group of strangers. The techniques would also be effective if one were trying to pacify an angry mob.

Third years who were 19 years ~~old~~ young deigned to teach 18 year old second-years how to introduce themselves properly to a group of strangers. The script was simple:

My name is [INSERT NAME HERE].

I am a 2nd year student at the Department of [DEPARTMENT] at IIT Kharagpur.

I live in Nehru Hall and my hobbies are [HOBBY 1, HOBBY 2].

The reader might well want to point out that this script makes a lot of sense and conveys information about the speaker to the audience concisely. I wholeheartedly agree with the reader. Although, it would be wise on the reader’s part to reserve judgment on this whole exercise until after I have related the instructions that came with this script regarding how it should be delivered.

The preachers preached, the tenet of utmost importance when introducing oneself was to be confident. Confidence being one of the non-verbal parts of communication, one must take extra pains to verbalize it. To this end, the preachers implored each second-year in the audience: When introducing yourself, SHOUT AT THE TOP OF YOUR VOICE.

After listening to a demonstration, I would offer an alternative characterization which will be easier to imagine if you are an avid horror movie watcher: SCREAM LIKE THE CHARACTER WHO HAS JUST SEEN A GHOST.

Once this tenet was revealed, the time was ripe for some hands-on learning: blameless exercises where a second-year student was asked to stand-up and deliver their self-introduction. If the student stumbled on any part of their introduction, the preachers asked him to start again, by shouting loudly at him. The preachers shouting makes complete sense as they were only being confident and not chastising the student in any way at all, in fact they were being very supportive, evidenced by both their mocking and their constant interruptions.

The preachers had noticed what I have long noticed about confident people. When confident people talk, it is clear to the listener that (a) they have rehearsed what they are saying a few hundred times, (b) they never pause to collect their thoughts, and (c) they do not change their response depending on the audience. The well-rehearsed delivery with no hiccups makes the speech sound natural, unlike the pre-recorded mechanical voice without any emotion that one hears when computerized assistants like Alexa or Siri introduce themselves. The lack of any breaks in the speech indicates that the speaker does not need to burden their brain with thinking or introspection. And the constancy in content irrespective of the audience is the pinnacle of the egalitarian principle¹.

Further, the preachers preached, when introducing yourself, look every person in the audience straight in the eye. The naive student of human anatomy would quickly retort that this is impossible; that to achieve it one would have to unnaturally oscillate their head, tracing a semicircle with the tip of their nose. They would also add that moving your head around like an automaton would make the listeners uncomfortable due to the uncharacteristic juxtaposition of the confident, not-at-all automaton-like voice shouting at them and the robotic movements. The preachers contended that this was a natural motion. The aim of the tactic was straightforward: one should make everyone feel included. After the shouting that had ensued just minutes prior, no one in the audience wanted to draw the ire of the preachers by voicing their opinion. The congregation had fallen in line.

As the street magician aims to swindle onlookers out of their money by showing them a sleight of hand and convincing them of their certain victory when the dice is cast, a confident self-introduction can sweep the audience off their feet through these 2 rudimentary tactics. Thus, the preachers ended their sermon.

These tactics are controversial: many people in the world outside IIT Kharagpur² believe that they are absurd, counter-productive and would make one a laughing stock if adhered to! Astonishingly, even the voice in the back of my head agrees wholeheartedly with those people. How can such utterly opposite natures reside within my own person?

These sessions continued unabated for 6 days with more-or-less the same content. The sermon ended at 11 pm and all the second-years were tired from having to sit on the ground for more than 2 hours. On returning to their rooms, they promptly fell asleep, mentally counting their blessings at having the guidance of world-wise men who were willing to spend mind boggling amounts of time schooling them on basic soft-skills, such as shouting at the top of your voice, choosing hobbies that were beneficial to your hostel and why other halls did not have that undefinable quality required to win the GC³. The drowsy second-years would remember that this valuable tutelage was not being offered to their peers in grossly inferior “hostels” such as LBS, where the seniors were shirking their duty of conducting O.P and had the audacity to focus on their personal lives. These seniors were indifferent and did not want to play a major part in valuable and meaningful endeavors such as raising the esteem of their living quarters.

While this sermon was delivered to a congregation where participation was voluntary, more was about to come, and future sermons would go where the second-year was. The Common Room was seldom frequented by some parishioners and these people were missing out on essential knowledge. To remedy this, the next part of O.P. was conducted in a place which no one could avoid: The Dining Hall.

Programming note: This is the second part of a 4-part series.

Post	Date
The Curious Case of the Dweller Who Wouldn’t Move Out	10th May, 2021
The Sermon in the Common Room	11th May, 2021
Victorian-era Dining Hall Etiquette	12th May, 2021
The Midnight Assemblies	13th May, 2021

Why would you want to say omit or include things based on your audience?! “It’s elementary, Watson”. ↩
Apparently, there is one! ↩
A soft skill only when looked at in the really big picture. ↩

The Curious Case of the Dweller Who Wouldn't Move Out

2021-05-10T00:00:00+00:00

At the end of my first year at IIT Kharagpur, an engineering college in India, I packed up everything I owned into cardboard boxes, put them in the common room of my hostel and went back to my parent’s home for the summer vacation. 2 months later, in the second week of July 2014, I returned to my new hostel, Nehru Hall. I found that my room had not been vacated yet and was not ready for “check-in”. This was the beginning of hazing at IIT Kharagpur, affectionately referred to by the euphemism, “Orientation Program” or simply, O.P. I recount my experience of O.P in this 4-part series. This is the first part, The Curious Case of the Dweller Who Wouldn’t Move Out.

I studied Mechanical Engineering at IIT Kharagpur from 2013-2018. I lived in MMM hall during my first year, from June 2013 to April 2014. Then, I shifted to another hostel, Nehru Hall, in July 2014. I lived there until my graduation in May 2018.

At IIT Kharagpur, hazing happens at the beginning of the second year, from August to November. Most colleges haze first-year students right after they arrive on campus. Clearly, at some point in the venerated history of the 60 year old institution that I studied at, some wise soul realized that tormenting 17 year old freshmen, who were living away from their parents for the first time, was cruel and set out to rectify the arrangement. The wise soul’s motives were pure, and knowing that there were good people on both sides, he decreed that hazing should be postponed from 1st year to 2nd year. We owe a great debt to him, for his decree improved the lot of at least a few people. Namely, the exceedingly small percentage of people who drop-out of college after their freshman year.

In June 2014, I learned about the hostel that I would be living at for the forthcoming 4 years through an Excel sheet. Arriving back on campus a month later, I proceeded to my new accommodations on the third floor of Nehru Hall. I saw a set of government buildings, inexplicably dark and ominous. Walking through the ground floor corridor, I looked for labels that would lead me to my room, I found none. I would later learn that labeling was frowned upon; A true “Nehruite”, as the dwellers took to calling themselves, would know where his room was, he would not have to be guided in any way, shape or form.

I was going to share my room with 2 other second-year students, both of whom I had never met before. While walking up and down several flights of stairs in this unmarked building, I felt like the proverbial driver wandering around in a cavernous underground parking lot having forgotten where they had parked their car. Eventually, I arrived at my future room. The door was locked and no one could be found nearby.

This hostel had functionaries, government officials tasked with running the mundane day-to-day operations. Presuming that their responsibilities included assisting new occupants, I asked them what the appropriate path forward was, specifically, where I was supposed to spend the night. Instead of getting an answer, I witnessed something remarkable. The functionaries vanished into thin air, leaving behind a flurry of smoke in the shape of “We have to ask the warden. Come back tomorrow morning at 10 am”. I was left to my own devices.

The presence of these functionaries is a sight for sore eyes, like the oasis seen in the desert by a famished traveler on a hot summer’s day. Like the oasis, the closer one gets to them, the clearer one’s understanding of their (lack of) actual powers.

Facing the prospect of a night without a roof over my head, I frantically called my friends to find out what their plans were. Several of them had not yet returned. Those that had returned had found lodging with other friends. I even called up a nearby hotel, the only one in the surrounding few kilometers. Smartphones might be widespread in the urban world of 2021. But people were still amazed at the sound of a ringing telephone and scurried away from the sound back in 2014. Callers who had the good fortune of having their phone call answered by this hotel’s receptionist counted their blessings.

Venturing further out in my network of connections, I called up a Ph.D. student I knew. He had not had to change his lodgings during the summer and he was willing to take me in for the night. To him, I am eternally grateful.

The lonely and dejected “Nehruite”, after being left without a room by his new hostel, had found a place to sleep. He nevertheless felt proud of the amazing feats that his hostel had achieved in years past. He had learned about them from the noticeboards in his new hostel which prominently displayed them. Untroubled by the fact that he had absolutely nothing to do with these past feats, he claimed full credit for them. He saw the bright future he had in his new “hall”, which was incomparably superior to the mere “hostels” that were scattered all around campus. He scoffed at the voice in the back of his head that told him that all the hostels were identical and had no significance outside of being living quarters. This voice tried in vain to convince the Nehruite that he is connected to his hostel’s past as much as a traveler is connected to the past of a hotel where they have never spent a night. The Nehruite disregarded the nagging voice with ease and saw meaning that was revealed only to him.

A week went by. Every day during lunch time, I went to my room to see if the previous occupants had made an appearance. As I understood it, they were taking their sweet time to return to campus. I was mildly annoyed at their lack of foresight and at the indifference they had shown towards the plight of the future dwellers of their room. But I waited patiently for them to show up.

All the second-years who were supposed to move to this hostel were in the same situation and were living with their friends in uncomfortably small quarters. But they found consolation in their dreams. Every night, they dreamed about the heights their new hostel would take them to. Especially during Diwali, the Indian festival of lights and a time of general merry-making, when they would be standing on cots borrowed from their peers. These peers had happily given up this material pleasure and were sleeping on the ground during the cold winter month of November without any resentment towards the borrower clouding their heart.

The voice dwelling in the back of my head reared its head once more and posed a reasonable question this time. “Why had the functionaries not foreseen the eventuality of the stranded second-year student and the third-year student who refuses to move out?”, it inquired. It did not give in to my diligent attempt at driving it away. Indeed, the voice was energized by the fact that MMM Hall, my old hostel, had foreseen this possibility and asked me to move out of my room before leaving the campus. The voice proffered, “Perhaps your new hostel is run by people who don’t care about second-years”. I was astonished at this suggestion and quickly dismissed it as absurd and baseless.

I was further troubled by the voice because it had thought reasonably. This kind of rational thinking was a most undesirable quality when dealing with the things that happened inside Nehru Hall. I reminded myself that asking questions had repercussions. Questions have long been known to be a thorn in the side of bad planning and intentional indifference.

10 days had gone by. At 5 pm on the eleventh day, I made my daily pilgrimage to my future room and was elated to see that the room was unlocked and that one of the occupants had shown up. Seeing as he was a year older than me and world-wise beyond my imagination, I felt no need to explain my predicament to him. Alas, on this count, I had grossly miscalculated. Much as a horse standing in front of a river indignantly demands to be lead up to water, so did the previous occupant demand to know why I had shown the impunity to show up unannounced. This Curious Character advised me to be humble, to wait patiently, to allow the elders to do as they saw fit and not to question their wisdom.

Finding no purchase for arguments laden with reason or emotion with the Curious Character, I went back to my previous approach of waiting patiently for him to move out. I later found out that this occupant was one of the well-known delinquents in the hostel and it was not surprising that he had engaged in this strange power play by prolonging his residence in a room that wasn’t his. “Knock and the door will be opened to you”, says the good book. I was disheartened to find out that there is no verse about occupants not shifting out of their old accommodations when the period of their stay ends.

5 days after I met him, the Curious Character finally moved out. I moved in swiftly and spent the remaining few days in July and the whole of August getting used to my new situation: new living quarters, new roommates and new neighbors. Nothing untoward appeared to be on the horizon and life proceeded with much abandon. Things were about to take a turn, but I was unaware of the precise nature of the upcoming twist in the road.

Programming note: This is the first part of a 4-part series.

Post	Date
The Curious Case of the Dweller Who Wouldn’t Move Out	10th May, 2021
The Sermon in the Common Room	11th May, 2021
Victorian-era Dining Hall Etiquette	12th May, 2021
The Midnight Assemblies	13th May, 2021

A Critique of Context-Free Journalism

2021-05-08T00:00:00+00:00

Journalism is the act of taking facts and expert opinions, putting them together with the context required to make them meaningful, and producing a piece that consumers will learn something from¹. This is the basic framework I use to understand and evaluate news organizations. Newslaundry is an Indian subscription-based independent news organization that I started following with the hope that their “indie music” approach to journalism was the refreshing breeze that Indian news media desperately needs. My findings were disappointing, a recent episode of their daily news briefing podcast had a story which was read out with none of the required context. This symbolizes the problems media organizations face and have left unresolved. I will continue to read the work that Newslaundry produces, albeit with lowered expectations when compared to what I have for the New York Times, NHK or Bloomberg.

These are the three major stages in the reporting of news:

Discovering information: Something happens: a report is released, an expert says something in an interview, a research paper predicts that the world as we know it will no longer exist in 2012 etc. Wire services like the Associated Press have existed since the widespread use of the telegram in the late 1800s and their goal is to produce the “official” version of events, which will be used as a jumping off point by news organizations such as the New York Times or local papers that rely on them for national or international news
Gathering context: “Understanding what happened yesterday, requires you to understand what happened a week, a month or several years ago” (Matthew Yglesias, one of the founders of Vox). This is the crucial step in reporting the news and the best organizations do this well. They take different approaches to this problem: Bloomberg employs a bevy of columnists with expertise in various niche topics who publish informed, well-researched, opinion pieces. New York Times takes a neutral approach which relies on a reporting network that spans the whole world and employs people from every country. These reporters have a native understanding of the events that they are covering. Vox does this through meticulous research on wonky topics which are mostly centered on America but occasionally global. Everyone agrees that this part is important but a lot of news organizations simply ignore this step, resulting in pieces that readers scroll past.
Writing an article: The final step in the process is deceptively simple. It involves an ill-defined mix of the skills of a writer and the skills of an editor to produce an article that is self-sufficient and is long enough to convey the right amount of information. The final product is not always an article: It can be a video (a script + visuals) or a podcast (a script + answers from an expert or opinions from the host). Some organizations excel at this step: Magazines are particularly good at this because they have the ability to spend the kind of time that would appear unreasonable to news organization focusing on the “outrage of the day”². Bloomberg Businessweek’s feature-length article about Tim Cook, New Yorker’s profile of Apple designer Jony Ive³ and Guardian’s Long Read about freewill are brilliant examples of well-reported, beautifully written pieces that cover an open question and give the reader the space to ruminate about what is written long after they initially read the article.

What is Newslaundry?

Newslaundry started with the goal of being independent and subscriber funded and they have stuck to this model. There have been a large number of reported pieces in which their reporters have talked to people on the ground and broadcast these conversations as video features or as articles on their website. Newslaundry also has a really good Hindi section with articles that cover a whole variety of topics and I appreciate that they are doing this despite the limited audience for online Hindi news⁴.

I started listening to Newslaundry’s daily news briefing podcast “Daily Dose” around 6 months ago. A new episode is released every day of the week. Each episode is about 8-15 minutes long and contains the top news stories from India and the world. The episodes also inevitably contain one message placed somewhere in the middle which urges the listener to subscribe to their service (the podcast is free and can be streamed on your favorite podcast app). Most of the episodes also contain references to a reported piece or a ground report on the website and I tried to read as many as I could manage to get a sense for both the podcast and the written pieces.

The written pieces are thoroughly researched and well-written. I have found that most of the pieces don’t have the depth that is required to understand the historical context, but this is a pet peeve and doesn’t hinder the reader from understanding what is being said or why it is important. No piece that I read over the past 4 months stood out particularly either for the content or the writing and I have only one cutout from the website during that period. The stories that were being covered were solidly in the current affairs realm. Their value stems from the fact that no other news organization in India is covering these topics with this steady regularity and journalistic integrity.

Context-free journalism

In Episode 715 of the “Daily Dose”, I found an example of context-free journalism that left me rather disoriented and cross.

(01:53) Indigo has transported a total of 4142 Oxygen concetrators, weighing nearly 72,500 kg, across India. 2717 concentrators were airlifted from Thailand, China, Qatar, Hong Kong and Singapore; while 1425 were transported domestically between 36 airports.

…

The Allahabad High Court …

The story about the oxygen concentrators has no context. The story before it was about diagnostic testing advisories to states and the story after it was about the judiciary criticizing the UP state government. In particular,

The listener has no idea what to make of this number: 4142 concentrators. How many concentrators are there in India right now? How many were transported yesterday by other airlines? Why is Indigo the only airline that’s being pointed out here?
The listener can’t make sense of the other number: 72,500 kg. How big is this? Is it the size of a single airplane? Is it multiple airplanes? I don’t know what this number is even doing in this sentence. Ratios and comparisons put big numbers into context.
What does the detailed split tell the listener? Nothing. 4142 = 2717 international + 1425 domestic. Sure, that sounds good, but what does it mean? All I understood was that a majority of the concentrators were imported. But why were they imported? Why aren’t more being manufactured in India? How many were imported yesterday? Are more going to be imported tomorrow? … so on.
Why is this important? I know that there is a shortage of oxygen concentrators right now. But assuming that the listener already knows a part of the news is counter-intuitive for a news podcast⁵.

The segment left me disoriented and with more questions than I had. And I was cross about it because I believed that this was an important news story but I didn’t get the information I wanted.

This segment with more context would probably sound like this: “About N concentrators were transported yesterday by the 3 major airlines, A, B and C. Of these, X% were imported from Country D, Country E and Country F among others, while the remaining were manufactured in India, mainly in the cities of G, H and I. India has been facing a major shortage of concentrators during the ongoing Covid19 wave of infections. Experts say that Y% more concentrators are required to prevent adverse outcomes due to the lack of this life-saving device”.

Clearly, this is a cherry-picked example. I had been thinking about writing a critique of Newslaundry for the past couple weeks and this podcast was simply the trigger to write it today and not put it off. But this is emblematic of a deeper problem that is either misunderstood or ignored by the practitioners of this profession, journalism: The listener always needs more context than experts do.

Someone who has worked in the logistics industry would know that transporting 72,500 kg of cargo in a single day is a huge achievement, or maybe they know that it’s just a drop in the ocean and not worth mentioning at all. I don’t work in that industry and I have absolutely no idea what it means.

As a great recent counter-example of journalism which has just the right amount of context, I would present the episode about Aleksei Navalny on The Daily, a New York Times podcast. I have never been to Russia and I knew little about Russian politics or geography before I heard this episode. After listening to it, I understood what was going on, why it was important and what Navalny’s role in it precisely was. The context was vital in my understanding that. I can easily imagine this episode botching the presentation up by reducing the context just a little bit (such as naming the cities where there were huge protests but not mentioning their geographical importance, or the historical rarity of wide-spread protests in Russia) which would have left me confused and tuned-out.

Other Critiques

Newslaundry also does “media critique” in the form of articles on their websites and a YouTube show called “Newsance” which points out the problems in the way other news organizations covered news stories. I have found this part of their reporting problematic because it gave me the feeling that they were playing both sides of the news business: they are an organization reporting the news and an organization criticizing other news organization who are in that business. Some of their media critique blends uncomfortably into plugging the way that they covered the news story that is being talked about. I haven’t seen or read a lot of their media critique articles due to this reason. I would much rather rely on non-profit groups and think-tanks to play the role of the “media watchdog”(like Media Matters for America in the US) and have news organizations only report the news.

I intend to continue to subscribe to Newslaundry and listen to their daily briefings podcast. However, I will re-calibrate my expectations and expect less context and more current affairs news, which is valuable in its own right, but is less valuable than a fully-formed product with the appropriate context. In a media landscape over-run with propaganda machines like Times Now and Republic TV, Newslaundry is one of the few organizations that publishes a wide-variety of coverage, a large part of which is critical of the government.

The “something” is intentionally ill-defined: People are different and they expect different things from the news they consume. ↩
A phrase that is often used by Ezra Klein on “The Ezra Klein Show” when talking about whether journalists should cover “current affairs” or focus on evergreen topics. ↩
Archive retrieved 2020-12-29 ↩
This is a hunch. I don’t know that the audience is limited. If I find data out there which says that the audience for online Hindi news is actually larger than the audience for English news, I will update this section. ↩
I have left out “What is an Oxygen concentrator?” in this list. I don’t know what they are or how big they are etc; I was not expecting this kind of context from the beginning because my expectations were already lowered. If this was a popular podcast like NYT’s The Daily or a Bloomberg QuickTake, then I would have expected an explanation of what a concentrator, what it looks like and why it saves lives. ↩

Monthly Reading Recommendations (April 2021)

2021-04-30T00:00:00+00:00

The recommendations for April 2021 are in! This month, an unsettling article about the disastrous consequences of the intersection of a dance form and a journey of self-discovery is at the top of the list. Also, there’s an article about pandemics (everyone’s favorite topic since Jan 2020) and a profile of the author of a book that I did not like.

Dancer in the Dark (Aviv)

New Yorker (archived), March 30, 2020.

The story of a dancer who wants to perfect herself at the Butoh dance form and her relationship with her teacher, mentor and performer. The consequences are disastrous; the reader realizes in just a few paragraphs that the worst is yet to come. For the rest of the journey while reading the article, one awaits with trepidation for the hammer to fall.

Pandemics and the Shape of Human History (Kolbert)

New Yorker (archived), March 30, 2020.

The period before and after a pandemic has never been identical: Things will not go back to the way they were. There will certainly be some sort of “after” period.

New Yorker profile of Sapiens author Harari

New Yorker (archived), February 10, 2020.

Harari’s book Sapiens was bad: It was vacuous and fiction masquerading as non-fiction. This profile paints an honest picture of a bizarre conversation. Harari’s skill at doing good PR is commendable: He understood that a lot of people read bestselling books so that they can say that they have read them. He wrote a book with an abundance of hand-waving and marketed it as non-fiction. One instance makes it clear that Harari cares little about the content and even less about his readers:

In “Homo Deus,” Harari writes that, in 2014, a Hong Kong venture-capital firm “broke new ground by appointing an algorithm named VITAL to its board.” A footnote provides a link to an online article, which makes clear that, in fact, there had been no such board appointment, and that the press release announcing it was a lure for “gullible” outlets. When I asked Harari if he’d accidentally led readers into believing a fiction, he appeared untroubled, arguing that the book’s larger point about A.I. encroachment still held.

…

“I don’t know if it’s a true story,” Harari told me. “It doesn’t matter–it’s a good story.” He rethought this. “It matters how you present it to the readers. I think I took care to make sure that at least intelligent readers will understand that it maybe didn’t happen.” (The story has been traced to a Johnny Carson monologue.)

Come to think of it, Harari writes books that are a lot like a tweet: Lot of bold claims, most of them untrue, some of them made up and all of them certain to outrage the reader into talking about the book.

Pew Research Center: Shrinking Global middle class

Pew Research Center, March 18, 2021.

COVID19’s economic impact has been vast, this report quantifies the impact on a wide spectrum of people by considering World Bank growth estimates from January 2020 and January 2021. The difference between these two estimates is the loss to each income tier. Global middle class population went from 899mn to 1.34bn from 2011 to 2019, an increase of 54mn people annually; the pandemic erased a year of growth for the global middle-class. Quantifying the impact gives the reader hope, as we have identified who needs to be helped out in future economic stimulus packages.

New Yorker, December 23, 2019

Hazing: Motives and countermeasures: An abstract analysis

2021-04-11T00:00:00+00:00

The point of hazing is to make the victims of the hazing want to haze the next class of victims – Levine on analysts at Goldman

This got me thinking about my experience with bullying during childhood and hazing in college. I turned out “okay”: I escaped the worst of it by laying low. I laughed at the parts that I found hilarious. I ridiculed the people who were hazing me. I looked around and found (rather easily) some of their peers, who also ridiculed them and were incomparably better than them (in my estimation). I endured what I could find no way out of. And I enabled the subsequent classes of victims to escape it.

On graduating college and entering a new phase of my life, old bonds where cherished and new friendships were formed; belonging was felt and people cared. With the benefit of hindsight, I consider the question of hazing once more.

Why do people engage in hazing?

Think as I might, I am unable to come up with a single, satisfying justification for it. Here are some candidates:

Hazing is a power trip: Just as the immigration officer adjudicating the entry of thousands of tourists asks them questions and forces them to do things that don’t serve any apparent purpose¹ before stamping their passport, so can the seniors inside a hostel reinforce their power through hazing. They are big fish in a small pond; a perverse pleasure can be gained from harassing others. Safe in their beliefs that juniors can do nothing against this harassment, that institutions don’t have the right controls to curb it and ignorant of the extent of their soft power, seniors take full advantage of their position
Cycle of abuse: “I was hazed, and hence I shall haze”. Even among those who seem to have experienced the underlying nature of the world, this argument holds considerable purchase. Often, this argument is framed as “I was hazed and it is only fair for me to try it out and see if it makes me feel better” or “I was hazed and I think it made me more resilient. Thus, it is my duty to pass this on”. I can say unequivocally that this argument betrays a damning lack of identity, morality and first principles, the things that are eternally true irrespective of the situation
Hazing builds community: This is an elegant argument from the abuser’s point of view: They convince themselves that they are bringing the victims together; pushing them closer by putting them all through the same kind of hardship, playing the part of the proverbial “lunch lady” whom they will rant about in weeks to come. The formation of a bond that would not have existed in the absence of this external stimulus is unnecessary, and the bond thus formed will not survive long. The community that is supposedly built is merely an illusion that abusers who don’t have the stomach to face up to what they are doing conjure. Communities are built through warmth, empathy and companionship in the time of need; not through abuse for abuse’s sake.
Hazing builds character: The abuser engages in artful deception. They wash themselves clean of responsibility by declaring, “I can credit myself for the characters that I have built today”. I have heard seniors boasting about former students who “made” it in the real world, whom they admire and want to emulate; the glaring lack of self-identity is completely unnoticed by these admirers.
Hazing is a necessity for the bond between institutions and students: The presumption with this argument is that the “Hostel” or the “College” that one attends is an institution whose relationship with the students who live or study in it is tenuous, like the branches of a tree in autumn whose branches stand ready to snap at the mildest gust of wind. I think this is true: Hostels are not endearing places, they are not comfortable, welcoming or well provisioned for living a good life; the food is horrible and the hygiene one wouldn’t write home about. One might remember the time spent there with nostalgia and affection, one might even visit with family, but one does not want to go back for an extended period of time. If the bond is tenuous, why should one reinforce it through abuse? Indeed, the fact that one has to resort to abuse to preserve a bond that would break if left to it’s own devices should give the abuser pause: What is it that you are trying to achieve by prolonging the bond through such cruel means?
Hazing makes people resilient to future hardships: Yes, on the surface, I admit this is true. In the pedestrian sense, anything that has been dealt with before is easier to deal with in the future. If one has dealt with abuse in the past, then they will fare better at dealing with abuse in the present. Does this tell us that we should intentionally put others through hardship, so that they might be better prepared for the future? Of course not, that is absurd. As Seneca reminds us, the stoic aims to remain unaffected no matter what happens by rehearsing the worst-case scenarios in his mind; this exercise has nothing to do with his experience of those worse-case scenarios.

All these justifications leave something to be desired for. They don’t clarify the question. Reading through these again, I feel a false comfort arising from having enumerated some possible root causes; I want the reader to be wary of this: We are hovering near the periphery; but we have not found the root cause.

Why don’t institutions put a stop to it?

Functionaries and officials at every college and university know that hazing is going on, and yet they do nothing. It is well within their power to stamp it out; I am sure there are educational institutions which have rid themselves of it completely. Why does the average institution stand silently by?

Institutions don’t really care: This is not a dig at institutions; rather it’s a dig at adults in society. As one grows older, all institutions care less and less about them and expect them to lead an independent, transactional life. All around the world, discussions about the social safety net are reaching a fever pitch in the higher income countries; while the merits of old-age homes are discussed openly in middle-income countries. Ezra Klein’s June 2020 AMA on The Ezra Klein Show has a poignant quote about this: “From taking care of my son, a toddler, it has become exceedingly clear to me that we should treat everyone around us the way that we treat children: When I speak to him, I say the things that would benefit him. I don’t want to be vindictive; I don’t expect to be treated the way that I treat him; I am able to act with empathy without having to make a conscious effort to do so” (paraphrased).
Institutions do care, but they have better things to do: This will be familiar to people working in Agile teams where “priority” is assigned to every task and some tasks in the Backlog never get done because they were never “high priority enough”. Most government-funded colleges are not run very efficiently and they are not under any obligation to improve their efficiency. My belief is that colleges have a surplus of staff and an unwillingness to do anything, like governments. A good demonstration of institutions getting their priorities horribly wrong was the paving of a parking lot at the Mechanical engineering department at IIT Kharagpur; while the department did not have air conditioning in any of the classrooms or a decent canteen. Assigning higher priority to the former rather than the latter is a gross misuse of public funds and a misunderstanding of the majority’s needs. Further, like the members of the Select committee trying to understand why an expensive roof garden was commissioned by questioning a bureaucrat, it is impossible to understand this decision. Trying to understand it have left me only with futile frustration.
Institutions are delegating some of their work to willing seniors: This is a controversial argument; the gist is that institutions do want someone to supervise the new students who are coming into the campus and ensure that they don’t get into trouble through “too much freedom” (whatever that is); but they are unable or unwilling to perform this supervisory function and are delegating it to other students who are willing to take up this mantle and work for the institution for free. In return, these seniors are rewarded with positions in the “Student government” which gives them a false sense of importance which they use to legitimize their harassment as a “good deed”.
Society has given up and accepted the existence of hazing: This is the most defeatist reason. There are several things that we have just accepted, there is not much debate on the topic because the debates seem to go nowhere and the conclusions are fait accompli: “Yes, Institutions should put a stop to hazing.”, “Yes, Governments should enact legislation and seriously pursue punitive action”. Few have the incentive to become an activist advocating for these topics as their impact is low in the utilitarian sense: “Just endure it for the first year of college, it will build character and you will understand more about the real world”, or my personal favorite, “What you are going through is nothing compared to what I went through; so don’t complain” etc. The other two star examples of this are (a) Governments, everyone is perpetually unhappy with them, but at least they are passably good at their primary responsibility, maintaining stability, and (b) Economic inequality, everyone acknowledges the existence of economic inequality and its evils, the activists on the two extremes are screaming at the top of their voices for either the Free market or Socialism. In this cacophony of voices, everyone remains unheard and governments don’t have any incentive to take action as they will be unable to take credit or use it to forge a path to reelection.

How do you deal with hazing?

Moving away from the abstract, this is a question that I want to give a satisfying answer to. I used 3 strategies back in college:

Maintain a low profile: During the period that you are going to be hazed, make yourself non-existent. I spent as little time as possible around people I didn’t trust or suspected would haze me at some point in the future; this includes not becoming too familiar with them (there is a misconception that if you are familiar with the seniors you will not be harassed as much; I have found the opposite to be true). Eventually, everyone will get trapped in their net and this happened to me as my implementation of this strategy was not sound. When seniors went on rounds through the hostel floor that I was living on and screamed like banshees at juniors for disrespecting them by not assembling in the common room, I committed an error and made myself noticeable. I stood up to them and refused to go down to the common room saying I had to “study for an upcoming examination”². They were not having any of it and I lost that battle. Retrospectively, it occurred to me that I was reinforcing their power trip by giving them an opportunity to wield the power for real.
Do something worthwhile: This is a general suggestion that applies to all the time that one spends at college. During the period that you are going to be hazed, focus on something worthwhile that is not academics and not social media. Whatever it might be, it will give you the perspective required to look past the shortsightedness of the people hazing you and come to terms with how pointless their exercise is and how pathetic it is that they don’t see that futility. For me, this was reading. I read a lot of novels during this time (like Dickens’ Great Expectations) and managed to find solace in the struggles of the protagonists and their clear-eyed opposition of their fictional tormentors.
Find and participate in a group with seniors who ridicule hazing: At my college, the seniors who engaged in hazing were a minority of their peers. I found several of their peers who ridiculed them and let me vent my frustrations to them: What seemed pointless to me, they had already overcome. I see that they were playing the role of a manager: they were absorbing my anxieties and frustrations and reflecting back the wisdom that comes with experience and a broader worldview.

Footnotes:

Going through immigration at Munich in August 2019, I was asked by the immigration officer to show him that I had the 600 euros, that I claimed to have, for the vacation that I was about to embark upon. An interesting start to a vacation. ↩
I should have known better than to use an academic reason against people who barely thought about it and believed that anyone who did think about academics needed to be reformed. ↩

Notes and Review: Letters from a Stoic (Seneca) (Campbell translation)

2021-03-21T00:00:00+00:00

I found out about this book after Sreejith wrote a note about it. I knew vaguely about Seneca as being a philosopher of some kind, but I had not considered reading any of his work before seeing this note. Now, I have read the book and it is at the top of the list of books that I strongly recommend to everyone. In this post, I have put together some of the most insightful things that I think Seneca talks about in this book. This book is a perennial guide to living the life of a “wise person”: happy, contented, and prepared for whatever fortune might throw their way.

This book was written in the first century, some 2000 years ago. Going into the book, I expected to find some references to things that were simply outdated and ancient. But this was not the case at all: The maladies that Seneca says affect society have remained unchanged through a variety of events that we believe have changed the world, but which have only changed the presentation of those maladies, without getting rid of them or even altering the underlying anxieties that people tend to feel in society (envy, fear, contempt, etc).

This is a book of letters that Seneca rights to his friend and student Lucilius. In these letters, Seneca gives Lucilius direct advice about how to behave and how to lead a life that is contented and free of the anxieties that Lucilius or the people around him have probably been facing. There is a tinge of mystery as we get to see only one side of the communication between these two men, leading to the reader having to fill in the blank about what it was that Lucilius asked Seneca for counsel on. This mystery did not hamper my ability to learn from this book, and identify the “rules” that I can apply to my own life.

Seneca’s writing (and Campbell’s translation) is fresh and lucid. It is not the stuffy, intellectual writing of a philosopher who is trying to find arguments and connect dots that don’t deserve or need to be connected to gain something new. This is addressed in one of the letters where Seneca says how philosophy has been reduced to philology, the study of words and how modern philosophers (modern when the letters were written, i.e. 1-100 AD) were splitting hairs about what a word means or how a syllogism is inaccurate etc. This meta-theme, “what should philosophy do for the reader”, can be found in several letters throughout the book and it was one of the important things that I learned from this book.

Bonus: He has a really good sense of humor.

‘Mouse is a syllable, and a mouse nibbles cheese; therefore, a syllable nibbles cheese.’ Suppose for the moment I can’t detect the fallacy in that. What danger am I placed in by such lack of insight? What serious consequences are there in it for me? What I have to fear, no doubt, is the possibility, one of these days, of my catching a syllable in a mousetrap or even having my cheese eaten up by a book if I’m not careful. Unless perhaps the following train of logic is a more acute one: ‘Mouse is a syllable, and a syllable does not nibble cheese; therefore, a mouse does not nibble cheese.’ What childish fatuities these are! Is this what we philosophers acquire wrinkles in our brows for? Is this what we let our beards grow long for? Is this what we teach with faces grave and pale?

– Letter XLVIII (48)

This is the kind of joke I would not be surprised to find in a Jerry Seinfeld stand-up comedy act.

Recurring themes

There are 3 themes that recur through the book and are approached from various angles and when dealing with problems faced by various people.

Steel yourselves against misfortune

As a member of the Stoic school of philosophy, Seneca is obsessed with how the “wise man” would react to misfortune, be it in the form of persecution, natural disasters, war, the death of a loved one or even his own death.

His advice is deceptively simple: Prepare to lose everything you have to the whims of fortune and yet, continue to live a good life despite the loss.

I know about the popularization of this principle (“prepare for the worst, hope for the best”), but the missing piece in the popularization is the reason one should prepare for the worst. Why should you prepare for the worst, when the worst is highly unlikely? Why should you hope for the best, when that is equally unlikely? Wouldn’t the wise man prepare and hope for the usual? If one were to do that, given the utter randomness of everything, they would be well-prepared for most things and ill-prepared for the small minority.

Seneca explains the reason eloquently:

For what is there that fortune does not when she pleases fell at the height of its powers? What is there that is not the more assailed and buffeted by her the more lustrous its attraction? What is there that is troublesome or difficult for her?

– Letter XCI (91)

One can know very little about what will happen and to assume that the usual turn of events will continue for a long period of time is incorrect. In fact, the usual turn of events will only continue for a limited period of time and the worst thing that you can think of will always end up happening.

So, one’s goal should always be to remain unaffected in the face of the things that happen to them (good or bad) and the only way to remain unaffected by anything that happens to you is to foresee the worst possible scenario. The change in my thinking was about this goal. My previous conception framed the goal of “preparing for the worst” in the next part of the adage, “hope for the best”: I am hoping for the best because my goal is to get through whatever happens unscathed, not to remain unaffected irrespective of what happens. Once again, I hand the mic over to Seneca who explains what our attitude should be: (emphasis added)

This is why we need to envisage every possibility and to strengthen the spirit to deal with the things that may conceivably come about. Rehearse them in your mind: exile, torture, war, shipwreck. Misfortune may snatch you away from your country, or your country away from you, may banish you into some wilderness - these very surroundings in which the masses suffocate may become a wilderness. All the terms of our human lot should be before our eyes; we should be anticipating not merely all that commonly happens but all the that is conceivably capable of happening, if we do not want to be overwhelmed and struck numb by rare events as if they were unprecedented ones; fortune needs envisaging in a thoroughly comprehensive way.

– Letter XCI (91)

This might sound pessimistic to some readers. It felt like the pragmatic course of action to me.

Study philosophy to build a better character, not a better intellect

This is a meta thread of reasoning that runs through the book. (As this is a book with philosophical arguments, the philosophical argument for what the purpose of philosophical arguments are can be considered to be “meta”). Seneca is convinced that the purpose of philosophy is to reform people’s characters and make them better people and that it is not to improve the reader’s intellect or make them look well-read or intelligent to their friends through syllogisms, clever turns of phrase or “the other toys of sterile intellectual cleverness” (Letter CVIII).

The purpose of the philosopher’s audience is to “rid oneself of his faults and acquire a rule of life by which to test his character” (paraphrase from Letter CVIII).

This was an enlightening concept to me: I have dabbled in the study of philosophy. I have read articles about the core tenets of Nietzsche, Sartre, and Socrates. Admittedly, I did that because I wanted to give the appearance of knowing about the different schools of philosophy. I was not looking for something in their arguments that would help me become a better person. I wanted to get the general lay of the land without any deep idea about specific characteristics of the landscape. The closest I came to that “active” part of philosophy was watching the TV Show “The Good Place” and hearing the makers talk about it on a podcast. This was an incorrect approach to the study of philosophy. Indeed, I should have read Sartre because I wanted to understand his existential argument and think about how it could help me become a better person or update my reasons about what I should and should not do.

On this point, the letters give the reader hope.

It is easy enough to arouse in a listener a desire for what is Honorable; for in every one of us nature has laid the foundations or sown the seeds of the virtues. We are born to them all, all of us, and when a person comes along with the necessary stimulus, then those qualities of the personality are awakened, so to speak, from their slumber. Haven’t you noticed how the theater murmurs agreement whenever something is spoken the truth of which we generally recognize and unanimously confirm?

– Letter CVIII (108)

I believe that a person has come along with the necessary stimulus for me.

Side note: Philosophy gets a bad rap, it’s students and practitioners should elevate it’s name through their wholehearted embrace of it for the right reason: gaining the wisdom required to lead a life that is “better than the mob”.

Avoid shabby attire, long hair, an unkempt beard, an outspoken dislike of silverware, sleeping on the ground and all other misguided means to self-advertisement. The very name of philosophy, however modest the manner in which it is pursued, is unpopular enough as it is: imagine what the reaction would be if we started dissociating ourselves from the conventions of society.

– Letter V (5)

Don’t expect a change in surroundings to fix you

What good does it do you to go overseas, to move from city to city? If you really want to escape the things that harass you, what you’re needing is not to be in a different place but to be a different person.

– Letter CIV (104)

There is a lot to be said about the merits of travel and going to a new place. It makes you worldly; it puts you in a surrounding that you haven’t been before, reducing you to a child, lost and alone; it lets you see new mountains, landscapes and rivers.

But don’t expect the change in surroundings to change you, even if you don’t put in any effort yourself. I don’t know if a lot of people travel to get rid of anxiety in the present day. I have thought about doing this before: When I felt stressed out or suffocated in college, I would go to Kolkata for a weekend. If I was tightly wound for some reason, then the trip helped me loosen up but it didn’t help me get rid of the underlying cause of the stress or the suffocation. I was not surprised by the lack of efficacy; but I was also unable to articulate the reason.

The rule on this particular theme is hard to pinpoint and I think it changes depending on who you are and how much experience you have had. I am looking forward to thinking about what the rule is for me and how I can apply it in my life.

Not completely convinced

I am not completely convinced about one point that he brings up in one of his letters. He says that the recounting of past sufferings and one’s victory over them is a pointless exercise and serves only to be detrimental to one’s conception of the past, that suffering and one’s spirit.

What’s the good of dragging up sufferings which are over, of being unhappy now just because you were then? What is more, doesn’t everyone add a good deal to his tale of hardships and deceive himself as well in the matter? Besides, there is a pleasure in having succeeded in enduring, something the actual enduring of which was very far from pleasant; when some trouble or other comes to an end the natural thing is to be glad. There are two things, then, the recollecting of trouble in the past as well as the fear of troubles to come, that I have to root out: the first is no longer of any concern to me and the second has yet to be so.

– Letter LXXVIII (78)

I don’t strongly advocate for thinking or recounting past sufferings. But this particular point seems to be at loggerheads with something that Seneca advocates for: self-analysis to improve one’s character.

I shall put myself under observation straight away and undertake a review of my day – a course which is of the utmost benefit. What really ruins our characters is the fact that none of us looks back over his life. We think about what we are going to do, and only rarely of that, and fail to think about what we have done, yet any plans for the future are dependent on the past.

– Letter LXXXIII (83)

I will re-read Seneca’s arguments in these two letters and think some more about what he is talking about. I suspect that, in the end, I will agree with him (he was wise, after-all), I am not convinced at the current juncture about it though.

P.S. I have used Roman numerals here because Seneca himself lived under the Roman empire and uses these numerals are used throughout the book. I have put the decimal representation of the numbers in parentheses because I found those easier to write and refer to when talking about the letters in my notes.

An Unscientific Comparison of News Websites - Part 3 (Subjective Analysis)

2021-03-15T00:00:00+00:00

This series of posts looks at 13 news website home pages and compares them at a single point in time based on the content that they choose to host and the presentation of that content to the common user. This is the third post in this series and it looks subjectively at the look and feel of the news articles on each news home page. As I explained in the first post in this series, this part of the analysis is subjective and not based on any standards. I take a free-form “scoring” approach to rank the website on some predefined characteristics and then compare those scores. If your sensibilities do not match my own, this part of the analysis might not hold much value for you.

The clean design scoring definitely exposed something that I was vaguely expecting, but couldn’t recognize until I had completed my scoring: It brought the sites I like the most to the top, with some surprising exceptions in the final score.

Non-obscure link names for other parts of the site are important

The surprising exception that came to the top was CNN. I don’t hold any strong opinions about CNN as I don’t frequent the site very often. Looking at the screenshots again, I notice that the website outdoes other websites in the “Useful links to other parts of the site” section. Vox loses on this metric owing to the structure of Vox.com as a series of “verticals”, which are not very closely related to news and can be more accurately described as “Special projects”. (e.g. the effective altruism Vox vertical, Future Perfect).

The internet is powered by linking several other relevant pages on a highly trafficked page. This increases the likelihood of people reaching the homepage through search and then going on to other parts of the site that they find interesting. The Vox method of linking to obscure verticals is sub-optimal from this point of view. On the other hand, smaller websites do this very well. FiveThirtyEight does a very good job in leading people to other parts of the site. They have a simple navigation bar with links that are very vanilla: Politics, Sports, Science, Podcasts, Video. I have used this navigation bar several times, particularly when I want to listen to a podcast or watch something that they recently put out, without any clear expectation of what topic that might be related to. It is hard to argue that “Open Sourced”, “Recode”, “The Goods”, or “Future Perfect” are easier to comprehend. The Print and The Wire, 2 Indian new-age news websites, do a good job at this too, by sticking to traditional News beats for their navigation bar.

Most of the websites are well designed

This is controversial, because it is fashionable to rant about bad design and clutter. My experience with these sites, with Ublock turned on, was pleasant. None of the websites took an inordinate amount of time to load. The worst offenders were sites with moving elements like countdown timers or interactive graphs that don’t add anything useful to the user’s experience. Bloomberg’s ticker is the one element that I haven’t gotten used to despite having used the site for a while now. I am tempted to say that I don’t think it adds value for any kind of user, professional stock broker or otherwise, and that it is a skeuomorphism-based hold-out from the past when displays were too expensive to be large enough to show several stock quotes at once.

News18 was the strangest website on this particular point. More on this later.

Hilariously, Republic World scored a 10/10 on “Clean design” because they chose to put a single embedded video on their website (I was tempted to give them 11 or 15 for their surprising restraint on this absolutely irrelevant point, even as their daily news coverage hovers somewhere between “shouting match” and “I can’t believe this is news”). Of course, the website with the cleanest design is one that is completely empty.

Non-news CTAs are bad for the user

This one is a no-brainer. A news website shouldn’t mess with the reader’s experience with unnecessary ads of campaigns that Indian TV news channels are major sponsors / participants of (NDTV started this practice a few years ago with their “Save the Tiger” campaign which was widely applauded by animal rights activists and became the vogue thing to do if you are a news channel with a meaningful audience). Most of the websites I looked at have the “Subscribe” CTA. Given that these websites have very few avenues of making money if most of their users have ad-blockers installed, this is a necessary evil and must be forgiven by the responsible consumer.

As for links to Twitter / Facebook / foobarbaz, I doubt that these are required. I can find CNN on Twitter. If I want to mention NDTV on Twitter, I will type @NDTV and select the first suggestion from Twitter that has the verified check mark. I think most other users will do the same. I doubt that any Twitter user will go from NDTV.com to Twitter.com/NDTV and then tap on the Tweet button. My intuition is that at some point in the past, product managers of the NDTV.com website decided to add these CTAs to their site and that they didn’t run A/B tests to see if these new CTAs drove any engagement. Removing code is harder than adding it into a large repository, because anything that is in the repository quickly used in unexpected ways by other developers. This probably dis-incentivised them from removing these CTAs later (This is all rank speculation, pay no mind to it).

At last, we come to News18. This is the part of the analysis where I slightly started to lose my patience because News18.com is just not a good place to read anything. It faired poorly in the objective analysis because there was a lot of Clickbait and a lot of Entertainment news. Apart from the News articles, the site has nearly 10 other CTAs. These say things like “Watch Live TV”, “Download News18 App” (inexplicably shown on a desktop browser), “Follow” the site on social media etc. There’s also some sort of campaign that is asking you to take a pledge to save water today. There’s a strange “COVID tracker” where the word “COVID” is replaced by a rendering of the many-pronged “virus”. I seriously hope that you never have to tap on any link that leads you to this homepage, as one can only get more confused from the barrage of irrelevant information that’s presented on it.

Data

Spreadsheet (ODS)

An Unscientific Comparison of News Websites - Part 2 (Objective Analysis)

2021-03-14T00:00:00+00:00

This series of posts looks at 13 news website home pages and compares them at a single point in time, based on the content that they host and the presentation of that content to the common user. This is the second post in this series and it looks objectively at the content and classification of the articles on each web page. We look at the various websites and try to classify why some news websites are clearly “good” while others are “tolerable” at best.

Programming note: (Post 1: released yesterday, Post 2: this post, Post 3: March 15)

After counting and classifying each news article on the News home pages, I eyeballed the data and some graphs. I came up with three insights that I think are instrumental in making a website good, based on the kind of content.

Quality and quantity are equally important (for the most part, anyway)

I was surprised by the results, but then I realized that Times Now seemed to be winning simply because they had crammed a huge amount of news articles onto the available space on their website. To penalize the websites that put articles on their site as fillers for the available space, I added a negative weight to 2 categories: Entertainment articles get a weight of -1 and Click bait articles get a weight of -2. All other articles get the same weight (= 1).

If one looks at the weighted score, i.e. the sum of all news articles after subtracting the penalties for including entertainment and clickbait articles on the home page, the websites reorder themselves and the reputed websites clearly show their higher caliber.

Interestingly, even in this ordering, Times Now is quite high, scoring the same as the venerable Washington Post and slightly less than the paper of record, The New York Times.

The key insight here is that quality and quantity go hand-in-hand. Websites like Bloomberg and NYT put a lot of information on the home page, which in turn lets the user decide what path they want to follow into the website. They don’t clutter their home page with Entertainment or Clickbait. They give the user a wide variety of content and statistically increase the chances that a reader will find something of interest and stay on the site. Times Now chooses the same strategy but mixes in a few clickbait-y articles. If you want to put clickbait on your home page, be sure to cram it as much as possible with non-clickbait, quality content to retain the experience of the serious visitor. Don’t be a tabloid website is a succinct expression of the same thought (A sample tabloid website).

New-age websites like Vox and Newslaundry go for quality-only and put a select few articles on their home page. For instance, Newslaundry, the Indian new-age mediahouse, only puts 6 articles on the homepage. In my opinion, this is an inefficient use of expensive space on the homepage. Any potential subscriber who comes to the homepage must be presented with about 10-15 news articles from a variety of genres increasing the chances that they are able to find at least one article that will lead to them clicking and reading it. Obviously, the more time they spend on the site, the higher the chances that they becoming a returning visitor or a subscriber.

Websites cater to their audience

Looking at just the non-entertainment and click-bait articles, the target audience of each website can be guessed fairly reliably.

Bloomberg has the most stories in the Economy category, while CNN and Washington Post have the most stories in the Domestic and Government categories. NYT has a dedicated “Opinion” section on it’s right sidebar, denoting the exalted place occupied by Opinion columnists in the NYT world. Strikingly, new-age news websites don’t feature much variety on their website. Vox has a wide catalog and on their YouTube channel, they have explored a huge variety of subjects. But they have chosen not to highlight this on their homepage, deciding to stick to Domestic stories from the US and Opinion columns. Newslaundry and The Wire also stick to old-fashioned Domestic and Government stories, rather than using their autonomy to their advantage.

Also, it is clear that most websites based outside India don’t highlight sports on their home page at all. This is perhaps due to the tendency of sports fans to get their news from hyper-focused Sports sites, rather than plain-old news sites. The advantage of putting Sports stories on the home page is debatable for NDTV and Times Now. They should probably add a prominent link to their Sports section in the navigation bar and remove any Sports stories from their home pages.

None of the websites have too much clickbait

Looking at the Entertainment and Clickbait articles, it is clear that none of the websites have too much clickbait. This was a surprising finding.

As a percentage of the total stories on the website, the number looks even better. Even the worst website, News18, uses only 25% of the available space for these articles.

Now, far be it from me to declare that media houses understand that clickbait is annoying and that they are trying their best to balance the Click-Through Rate and Clicks Per Million dollar figure against their instinctive ideal to do good journalism and report on stories that the reader will find useful, especially when the reader next finds themselves in a voting booth. Fortunately, even the most pessimistic reading of these numbers leads me to believe that clickbait does not make up the majority of articles that these media houses publish on the Internet. This indicates at the remote possibility that the TV production crew at these media houses is more focused on producing sensational coverage when compared to the newsroom.

P.S. I am writing this article on 14th March, 2021, just as the Oprah interview with Harry and Meghan news story is picking up steam. All the major media houses have written almost identical news articles on the topic. There’s very little information left to share with readers who are just waiting for the other shoe to drop. The coverage of this story has been surprisingly low-key and levelheaded. Bloomberg ignored the story, while Vox and NYT published opinion and detail pieces about some of the words that Meghan used in the interview, while India’s NDTV published a snarky opinion piece from Shobaa De. It is unclear at this point which of the following scenarios will play out:

The story has not caught on yet
I am not checking the right places (Twitter, the town square where ignorant spectators have a lively, informed debate)
The story will never become too big because it’s just not as interesting when people are stuck at home and looking for an adrenaline rush (like the one from stock trading), rather than a gossip story, which they used to simmer down and get away from stress in the past
I am way off the mark and the story is not being reported in a “low-key, levelheaded” fashion

I am attached to option (3) above and would prefer it to be the case.

Data

Spreadsheet (ODS format)

An Unscientific Comparison of News Websites - Part 1 (Motivation and Methodology)

2021-03-13T00:00:00+00:00

This series of posts looks at 13 news website home pages and compares them at a single point in time based on the content that they host and the presentation of that content to the common user. This is the first of three posts for this series. It explains my motivation and the methodology that I used for this comparison. I took screenshots of the above-the-fold content on 13 popular news websites at a single point in time. Then, I counted the total number of news articles on each site and classified each one into various categories. I also did a subjective analysis of the site’s look and feel by ranking them between 1 and 10 on 4 metrics (I came up with these metrics using my expectations from news websites as a guide). Once again, this is an unscientific comparison study.

Programming note: (Post 1: Current post, Post 2: March 14, Post 3: March 15)

This is a series of 3 posts. This is the first post in the series. The other two posts will be released tomorrow (March 14th) and the day after (March 15th).

Motivation and Methodology (this post, released March 13th)
Objective analysis: Results and Insights (March 14th)
Subjective analysis: Results and Insights (March 15th)

Motivation

I have been visiting a few news websites regularly for the past 3-4 years. The New York Times website was the first one I started visiting regularly, after subscribing to the “Daily Briefing” email and finding interesting articles on it almost everyday. With the rise of podcasts, I come across new articles that authors and hosts talk about on podcasts, which in turn introduces me to new news websites (that is how I found out about FiveThirtyEight). Some websites are very good and I return to them often; others, not so much. This gave rise to the key question that this analysis tries to answer: What makes a news website good and worth returning to?

I think there are two parts to the answer.

First, at considerable risk of stating the obvious, the website should contain factual, reported, interesting news content. I add the epithet “interesting” here because news websites are not wire services (like Reuters or the Associated Press) which simply report the latest fact without any context. They serve the role of an “intelligent aggregator” sitting between the primary source of information (wire service, polling company or research paper) and the layperson. Nearly always, the reader will want to be caught up to the information already known and how the information being reported affects the story (Note that I saw “nearly always” here because some of the stories need little context. On the 2021-03-13, everyone in the world knows about COVID19 and hence one needs little or no context in a story that explains a new vaccine development). Sometimes, the layperson might want to dig further into a topic, the website should encourage and facilitate this desire.

Second, the website should present content in a pleasing, easy-to-read manner. This is clearly subjective: one person’s easy-to-consume is another person’s bland or unimaginative or lazy. Here, I should clarify “easy-to-read” is not talking about the language that is used (e.g. should be at 8th-grade or lower reading level etc). Rather, it denotes the aesthetic presentation of content and the consistency and restraint that the website shows in using technology to deliver news to the user. If there are a huge variety of colors that don’t conform to a single color palette OR if there are multiple fonts on the website, that makes the website less “easy-to-read”. On the other hand, if the website uses one of the standard presentation formats, with a single font and a few meaningful font sizes and has no adhoc elements added for special events, that makes it more “easy-to-read”. To give this series’ readers the utmost flexibility, I have taken care to ensure that the Objective and Subjective analyses are distinct in this series. If your tastes are not in line with mine, please ignore the subjective part of the analysis.

Screenshots

I chose 13 popular news websites and took a screenshot of their home page at 1 pm UTC on 10th February, 2021. I added Vox to the list some time later, so that screenshot was taken at 2 pm UTC on 7th March, 2021.

The screenshots were taken on a sufficiently wide screen, which ensured that most websites didn’t use all the horizontal space available
I opened the websites on Firefox with the Ublock extension for blocking ads and the Ghostery extension for blocking analytics enabled
I didn’t look at any “popular website” ranking to arrive at this list and simply chose the top websites that were at the top of my mind when I thought of “news website comparison”
You can check the screenshots that I took through the Screenshot column in the following table
I have inadvertently included an almost identical number of American and Indian websites

News website	Country	Type of organization	Screenshot
Bloomberg	US	Large, traditional	Image
CNN	US	Large, traditional	Image
NDTV	India	Traditional	Image
News18	India	Traditional	Image
Newslaundry	India	New-age	Image
New York Times	US	Traditional	Image
Republic TV	India	New-age	Image
Reuters	US	Wire service	Image
The Print	India	New-age	Image
The Wire	India	New-age	Image
Times Now	India	Traditional	Image
Vox	US	New-age	Image
Washington Post	US	Large, traditional	Image

Traditional news organizations are the ones that have been around for a relatively long period of time and have an approach to news that has remained largely unchanged for the past 40-50 years, even as the medium for delivery might have changed considerably. The New York Times and Washington Post are the stalwarts in this section.

New-age news organizations are those that don’t depend on traditional funding patterns of news (e.g. advertisements, product placements), instead choosing to sell subscriptions or taking a radically different approach to monetization (e.g. partnership with companies like Netflix). Vox is the flag holder for this group.

I have also included 2 Indian websites, Republic TV and Times Now, which are generally known for their entertainment-disguised-as-journalism approach to the nightly news. I wanted to see if these had bad websites or if their TV production was more sensational by some basic necessity of that medium. I found some interesting results; particularly with Times Now which ranked high due to meaningful news on the website and hardly any clickbait stories (This points to the remote possibility that their production team for TV news is doing sensationalist journalism and that is not representative of their newsroom).

Objective analysis: Count and classify

To analyze these sites objectively, I counted the total number of News stories on each website and classified each one into one of the following 8 categories:

Domestic
International
Government
Economy
Sports
Entertainment
Clickbait
Other

“Domestic” refers to the country of origin for each news organization. i.e. the same article about a policy decision from the Indian government will be classified as “Government” for NDTV whereas it will be classified as “International” for Bloomberg.

“Entertainment” is the section which would contains news about the film industry or celebrities. I have classified news about arts and music into “Other”. This is in line with my own sensibility that the film industry being a relative newcomer to the world of art.

Opinion-pieces and podcasts are classified in the Other section. Some websites have dedicated “Opinion” reporters and they are given prominent positions on their websites (Bloomberg, NYT).

After the counting and the classification, I calculated the weighted score for each website. During this calculation, I gave a negative weight to “Entertainment” (-1) and “Clickbait” (-2) stories as these are (in my opinion) intrusive, don’t serve any clear purpose and don’t educate the user. In particular, stories like this one are objectively bad for the reader, because they serve as a distraction.

As the attentive reader might have noticed, the weighted score doesn’t have any upper limit. Hence, one must resort to subjective comparisons of different parts of the score.

Subjective analysis: Look and feel

To analyze the look and feel of these sites, I scored these websites on a scale of 1-10 on these 4 metrics:

Clean design
1. A widely debated term, getting it right is impossible. As a reference, I believe that the front pages of DuckDuckGo, YouTube, and the repository page of GitHub follow the principles of “Clean design”
2. On the other hand, the Booking.com hotel page is a prime example of a utilitarian page which doesn’t adhere to the “Clean design” ideals and crams too much functionality into a single web page (e.g. Learning that a room at this hotel was booked 2 minutes ago helps Booking.com get to the viewers “FOMO” and helps them make a sale; it leaves the viewer in a decidedly worse state when compared to their self before they arrived at this web page as the only two options left to them are to be swindled out of their money or to leaving the page with the anxiety that the room will not be available a few minutes hence)
Distractions
1. Print newspapers are the holy grail: They have distractions in the form of advertisements but these are placed in well-defined blocks and easily skipped / ignored.
2. Modern websites have taken to putting up marquees, counters, count-downs to events, interactive graphs, icons and a lot more on their homepages which ostensibly serve to let the viewer know that they are on a “dynamic” website. None of these are related to news in any way, do not make the presentation any better by their conspicuous presence and don’t add any value for the common user
Useful links to other parts of the site
1. News home pages should serve as jumping off points which can be used by readers to reach other parts of the site that they are interested in. If the website doesn’t provide this functionality, then people will have to use external search engines
2. On a philosophical level, the lack of useful links on the home page indicates at a low expectation of longevity from each viewer
Non-news calls-to-action
1. Things that lead to affliate websites, ads, other websites run by the same company, CTAs that go to social media etc are non-news CTAs
2. The “Subscribe” CTA is a key part of the website and I don’t consider that a non-news CTA if the news organization runs on the subscription model

In each case, 1 is bad and 10 is good. i.e. 10 denotes extremely clean design, the least amount of distractions, several useful links to other parts of the site and the least amount of non-news CTAs. This part of the analysis gives us a score between 4 and 40 for each website. This score tells us how good the website feels to a user who has the same expectations from news websites as my own

Note about the scoring: After I completed the scoring, I realized that the scores were on the high end of the spectrum as they varied between 22 and 40, out of a maximum score of 40. This was probably because my scoring was accomodative and I didn’t take a stricter view of the minor quirks in some web pages. A more detailed analysis which looks at each component carefully and preferably done by someone who has real UX and Design experience would lead to a better Subjective analysis. Nevertheless, I decided to go with what I had for this iteration of the analysis.

Financial Markets are Complex

2021-02-18T00:00:00+00:00

The Gamestop stock started 2021 at a modest $17.25. It then went up to $347.51 on 27th January. It went back down to $53.50 on 4th February. It closed on February 17th at $45.94, about 2.66 times it’s price at the start of the year. Millions of words and thousands of column inches have been devoted to explaining what happened, why it happened, and who was hurt. I wanted to write about a few peculiar things I noticed in the past few days that were the bizarre side shows to the main event: politicians (and Elon Musk) posturing for publicity by saying things about other things that they know nothing about, truth not prevailing no matter how much effort is spent on explanation, general disapproval of complexity in a system and societal conviction that the stock market is “rigged”.

Politicians posturing for publicity

Politicians in the US tweeted about this and claimed to fight for the “little guy” and question Robinhood. Surprisingly (for me), they didn’t do their homework and were caught saying things that made little sense. Elizabeth Warren was a notable exception and no wonder; a veteran policy maker with deep knowledge of the stock market, she made a very good case in her letter to the SEC. Elon Musk is not a politician, but his posturing for publicity is no different from other politicians, and he was able to fuel the mania effectively.

This might not come as a surprise to some readers. Politicians say outrageous things because it gets them air time. Reality shows are a close parallel that might help us understand this phenomenon and teach us how to correct our reaction for this “posturing bias”. Indian reality shows like Splitsvilla and Roadies often have contestants whining about another contestant “grabbing footage”. These reality shows are inspired by American reality shows like The Bachelor and Roadies USA. The motivation for most people who go on these shows is to become famous or to make it in the Indian film industry. Their most effective way of doing that is by doing something outrageous and getting noticed by producers and casting directors who are skimming through thousands of portfolios to find the person who will play the lead or supporting role in their next venture. These shows are notoriously bad at turning contestants into actors: The only person who can be considered a success for the Roadies => Film industry path was Ayushmann Khurana, in the more than nearly 15 years that these shows have been running on television. This low success rate has dampened the popularity of these shows, but they continue to live on as zombie versions of their old popular selves.

One of the effective strategies to correct for posturing bias can be seen in action in reality shows, when other contestants wake up to the footage grabbing behavior of one of their co-contestants and begin to isolate them from their plotting. This inevitably leads to a loud showdown between various factions of contestants and ends with some people getting “eliminated” from the show. Now, irrespective of whether the footage-grabbing contestant left the show, this sudden blow-up changes the dynamics and the group re-aligns around some other contestant. As a viewer, this is a great “reset point”.

We can apply this strategy to politicians: tune out their opinions and look at their past actions. This is hard to do because the number of mediums that politicians can use to reach people has grown exponentially; talking heads on TV can talk about any subject for hours on end because there is no real expertise that is required for their job and their lack of such expertise allows them to talk with unflinching confidence even when they know nothing about the topic. Most media coverage does not cover what politicians did, rather preferring to cover the process that they were using or what groups they are aligned with or what their latest speech said.

So, if one wants to figure out what a politician actually did, one really has to dig into their legislative record. Personally, I have found this process of digging around to find out how a politician has performed to be rewarding as it has helped clear up my misguided beliefs. (E.g. Since Kejriwal got elected as the Chief minister of Delhi about 5 years ago, I was a staunch believer that he was an ineffective leader and that he had failed the people of Delhi. I believed this without any evidence or lived experience. When he won re-election, I was forced to challenge my views. This lead to the discovery that I had been completely wrong in my belief)

The process of researching the politicians past actions and figuring out how much of what they talk about has turned into action is time consuming and requires you to break down some of the internal barriers that give information that is learned earlier more importance that information that is added later on. Nevertheless, this is an effective method to reject propaganda and attention-seeking statements.

Truth does not “eventually” prevail

Contrary to popular belief that truth will eventually prevail, the past 5 years have been proof that that is not a feature of the world that we now inhabit. With respect to GameStop, this was clearly in the information about clearing houses demanding collateral from Robinhood not getting through to anyone, even people who were closely tracking the developments. When the mania started to die down and politicians shifted into their “posture by claiming to do something” mode, contention started going around about payment for order flow and what it meant. Everyone disregarded the explanation that people who had worked in the financial markets were trying to communicate to others and instead started rallying against it, despite it being the reason that brokers like Robinhood were able to give trading services at 0 trading fee to their users.

This problem is harder to effectively solve; while one might be tempted to offer up “Do your own research” as a strategy, I am skeptical that the truth can actually prevail when we are talking about binary representations (truth/lie) of complex situations. Particularly, depending on the case, the two situations could be so close to each other that it is hard for a rational observer to parse the facts and reach the same conclusion every single time. 13 years removed from the Economic crisis of 2008, we can say with a fair amount of certainty that the folks at Standard & Poor were not rating the financial instruments they were seeing fairly. But this is simply one piece of the puzzle; within the complexity of the financial system, several other players and forces are hidden and one person would not be able to understand the full scope despite spending unreasonable amounts of time reading / researching the topic.

Although, that is not to say that I am defeatist on this. The strategy that I have found useful is to limit your media diet: Focus on secondary sources that you trust as sources of verifiable information and depend on primary sources as sources of detailed information about the topic. E.g. News reports on Bloomberg would be sources of verifiable information about the state of the economy in a given country, but if I wanted to find out about unemployment or the reasons for unemployment in a particular sector of the economy, I would have to read research papers, surveys and government reports which can be considered primary sources because they are much closer to reality than News reports.

General disapproval of complexity

Occam’s razor provides an articulate argument for rejecting complexity:

Among competing hypotheses that predict equally well, the one with the fewest assumptions should be selected – https://github.com/icyflame/awesome-social-science#occams-razor

At my day job as a software engineer, I eschew complexity at every step along the way. I have heard that this is a good principle when designing large, distributed systems which tend to have several points of failure and an increase in complexity compounds the various causes of failure and prevents effective investigation, despite one’s certainty that there is a problem that needs to be fixed.

Fortunately, the real world is not a software engineering design problem. There is no limit to the complexity that can be attained with fairly simple systems whose dynamics are linear and well understood in some cases, but can quickly become non-linear and chaotic (E.g. pendulums that enter a chaotic state when the forcing frequency is slightly different from the natural frequency). So, one of the contracts that we must maintain with the real world is the active acceptance of complexity in the systems that we interact with: bureaucracies, economy, healthcare, education. All of these systems includes hundreds of thousands of people and to expect any of those systems to be something that can be explained in a news article, YouTube video, or book is irrational.

The Gamestop episode has shown that complexity is not palatable to most people. In particular, the minority of minds that are vulnerable to conspiracy theories look at the workings of a complex system, that is way beyond the comprehension of a single person and see a vast conspiracy that is making the system do things that it was not “meant” to do (like a large bureaucracy with intertwined departments). Admittedly, they have little evidence about what the conspiracy is or who is pulling the levers, but they have a convenient and rational escape for this line of reasoning: The system itself is keeping the “truth” from observers because it is in the system’s interest to do so. Once the initial threshold of irrationality is crossed, the line of reasoning becomes rational and easy to accept.

Once again, this problem is hard to solve, even on a personal level. I am not sure how to correct for this anti-complexity bias when the problem is known to be complex (E.g. Global warming). The urge to assign blame to invisible sources is quite high and everyone has to struggle with the uncomfortable state of accepting that a system that one is participating in is beyond one’s comprehension or accepting the “simpler” solution of a conspiracy that explains the complexity as an orchestrated illusion. I advocate for the former.

Conviction that the market is “rigged”

This amazing quote from a recent article perfectly summarizes this conviction:

“It’s another episode similar to those past ones of the public feeling like there are multiple things wrong here — not really knowing what is exactly wrong, but just feeling like something is not working. It’s just a general popular sense that a system wherein this kind of scenario can come to pass, just fundamentally doesn’t work for the public and it is ‘rigged,’ or something else, but they know something is wrong.”

– Graham Steele, senior fellow at the American Economic Liberties Project – https://www.businessinsider.com/reddit-wallstreetbets-traders-failed-war-wall-street-proves-system-rigged-2021-2?amp

What happened with Gamestop is a variation of the widely known “pump-and-dump” scheme: People buy a stock they know is worthless, pushing it’s price to a value that doesn’t make any fundamental sense. In a classic pump-and-dump scheme, the people who pushed the value up will dump the stock at a peak point that they pick. In this scheme, most of the buyers have held on to their stock. Their reasons seem to be a mix of “It was fun and I wanted to be a part of it”, “It’s like gambling. Also, YOLO (You Only Live Once)” and “I get Reddit Karma when I post a photo that shows losses”. Putting that aside for a bit, what happened with Gamestop is not a revolutionary insight into market operation. It is a mundane methodology which can be used to push the price of any instrument up (Musk is at it with BTC: BTC is at a new all-time high, again).

That said, now, what is the argument behind the “market is rigged” thesis? As far as I can tell, it is a variation of the argument that retail investors are not as good at making money as institutional investors. Colloquially known as “the house always wins”. I don’t think this is true. I will analyze it from two points of view: Process and Resources.

Process: Retail investors, who have money that they want to invest into the stock market, can invest just as institutional investors do. Buffett’s principle of “Value Investing” is one theory of doing that: One can look at a public company’s financial reports, recent news, governance structures etc and decide whether one wants to invest in it. The process is not different for institutional and retail investors. Now, retail investors are unable to do some things like “shorting a stock” or purchasing government bonds from foreign governments directly. But these are mostly related to a particular instrument and not a feature of the market itself (i.e. Institutions don’t want to let retail investors short stock because if the retail investor doesn’t pay or is unable to pay to buy the stock back at a later date, the institutional investor will be on the hook for this money).

Resources: Institutional investors have a major advantage at buying and selling on the stock market: It is their job. People who work at investment banks and hedge funds have spent several years understanding how markets work and how to turn a profit using that knowledge. To begrudge them their expertise seems unwise and petty. In terms of resources, they clearly beat the retail investor: A retail investor clearly has less time and they have much less money to take the risky bets that institutional investors can sustain. I don’t think this is anything even close to the market being rigged. In fact, it simply points to the fact that the market rewards expertise and that capitalism rewards capital.

For most retail investors, the suggested investment strategy is to invest in “Index funds”. Now, considering that most normal people will do something like that, I think the “market is rigged against normal people” thesis loses even more ground because it is rarely in the interest of average investors to pick individual stocks and invest in them, rather than investing in Index funds (Not investment advice).

(A useful example for this might be to ask yourself if something else is rigged, by this same logic. For e.g. Someone who wants to play Tennis or Golf will have a big advantage if their parents have the time and money to go to various tournaments, hire a good coach, buy the right equipment, regularly drive them to practice sessions and so on, their chances of success at the sport go up exponentially. Does this mean that the sport is rigged against people who don’t have these resources? In my opinion, the answer is YES; I just don’t feel as strongly about leveling that playing field as strongly as I feel about something like poverty in various countries.

I would like to end this analysis of the various players in the landscape today with a quote from the journalist Matt Levine. His newsletter “Money Stuff” has been an incredibly valuable resource as I have tried to understand what really happened, and why it is only history repeating itself and not some unprecedented event.

I don’t think, however many days we are into this nonsense, that GameStop is a particularly important story (though of course it’s a fun one!), or that it points to any deep problems in the financial markets. There have been bubbles, and corners, and short squeezes, and pump-and-dumps before. It happens; stuff goes up and then it goes down; prices are irrational for a while; financial capitalism survives.

– Matt Levine, the “Money Stuff” newsletter – https://www.bloomberg.com/opinion/articles/2021-02-01/gamestop-gme-is-old-news-after-reddit-discovers-silver-futures

P.S. 1: One of the articles linked above is a dailymail.co.uk article. I couldn’t find a similar article on other websites so I decided to bite the bullet and link an archived version of that article. If possible, you should ignore the bar on the right side on that page or consider opening the page in the Reader mode on your browser. (Daily Mail is a gossip website, similar to TMZ)

2020 Was Not "Unprecedented"

2021-02-12T00:00:00+00:00

“2020 was an unprecedented year because …”. All of us can find several phrases to complete that sentence. Versions of this quote have been circulating for more than a year. In particular, creators on YouTube and News announcers love this quote; every turn in the economy and the things they are able to / unable to do in their lives is “unprecedented”. In her book “This Time Is Different”, Reinhart makes the argument that there is a compulsive urge to claim that the present is completely different from the past, that there are no similarities and that whatever challenge / victory came yesterday is different and more significant than something that happened a 10 years ago. Reinhart rejects this practice of turning every insight into an epiphany and points to the fact that the same type of financial crises keep happening, over and over again, to demonstrate that the present is no more special than the past.

A lot has happened in the first 20 years of the 21st Century: devastating terrorist attacks in several major cities across the world (New York, London, Mumbai, Hyderabad, Boston, Paris), wars in several countries in the Middle East, the recognition of a possible climate crisis that could have very bad outcomes for countries that don’t have the resources to adapt, encroachment into sovereign countries by ambitious states that aim to expand their sphere of influence, the rise of the Internet and the substantial increase in Internet penetration across the world.

A lot happened in the 20th Century: 2 World wars, a deadly epidemic (Spanish Flu), a prolonged period of heightened tensions and anxiety, decline of monarchies and the rise of democracies, collapse of the colonial powers and freedom in their colonies, the discovery of significant crude oil deposits in the Middle East, rise of the personal computer, numerous healthcare breakthroughs, highly integrated world trade and an increase in the average number of allies of a country (2.5 1900-1950, 10.5 1950-2000).

A lot happened in the 19th Century: 17 major wars between 1815 and 1914 in Europe alone, the persecution of colonies by colonial powers, Japan went through the Meiji restoration and expanded it’s empire into Korea and parts of China, Jane Austen wrote some of her best novels.

A lot happened in the 18th Century: 33 major wars between 1715 and 1815, the founding of America as a constitutional democracy, …

These are just a few of the events from memory, references in the books I read over the past few months, and my experience since 2005. Historians would be able to lay out these centuries in a more logical, articulate manner (They would also be able to remember more things that happened in the 18th century, where my memory has not served me well). My argument here is that things happen all the time. They happened 300 years ago, and they happened yesterday. One should undoubtedly look at the things that are happening in one’s life as events from which unique insights can be garnered, but one should not make the mistake of believing that these events are unprecedented.

My favorite example of this is the invention of the Printing press by Gutenberg in the mid-15th century (1446-1450). Between the late 15th Century and the late 16th Century, book prices went down 90%: In about 2 generations (life expectancy was low), the price of books had become 1/10th in real terms (i.e. removing the impact of inflation on the prices). Also, recent economic impact studies have shown that the impact of the printing press on society was larger than that of the personal computer between 1977 and 2004. The major difference being that the PC’s impact was much faster (40 years as against the 140 years it took the Printing Press to be propagated across the world). I imagine myself being 15 years old in 1450, when Gutenberg made his breakthrough. In the next 40 years, book prices would have gone down by nearly half. That sounds very much like an unprecedented event, but I hesitate to give that tag to anything. Especially when dealing with the present, one should take special precaution as they have not yet had the luxury to analyze the events in retrospect.

The other misconception that I have seen thrown around is that the period since the end of WW2 in 1945 has been relatively peaceful. In objective terms, this might have been the case as life expectancy has gone up and infant mortality has gone down across the world. Rosling makes a compelling case in his book, Factfulness. But aggregate statistics seldom serve as accurate reflections of all the people in the world the world (even though they are useful to represent progress on a timescale longer than 75 years).

The Cold war was not actually “cold” and the post-WW2 period was not peaceful for the people in Korea, Vietnam or Congo. The US and USSR acted as proxy powers in several countries in Africa, Asia and the Middle East (CIA orchestrated coups). The type of warfare changed from trench warfare, the norm during WW1 and WW2, to guerrilla warfare, required to root out the insurgencies of the 20th century. Nevertheless, the facts remained unchanged for people in the countries that were stuck fighting a war: death, displacement and destruction.

On the economic front, the 20th Century also saw some major upheaval: Germany was reunified in 1990 after 4 decades of separation. The reunification was followed by a decade of European soul searching, as various countries in Europe tried to signal their commitment to the bloc; this effort culminated in the formation of the European Union and the adoption of a common currency in the early 2000s.

India’s economic policy saw major changes in 1991 and the economy was opened up to foreign investment and products, the monopolies in the automobile and electronics industries were brought to an end. The impacts of this policy change have been long reaching. State-owned enterprises that were created after the globalization of the economy (like BSNL) have struggled and continue to face a government that wishes to privatize these businesses. This indicates another major limitation from not being able to analyze the event in retrospect: One is unable to properly analyze the side effects.

Before assigning the dubious distinction of “unprecedented” to something, it is worth pausing to think about whether this take would be valid a year, 5 years, a decade or a century from now. The telegraph shortened physical distance like never before; the Internet blew the telegraph out of the water. I am convinced that the sober approach of not saying that something is unprecedented is the way to go when analyzing the present.

My experience with partiality in school sports

2021-01-30T00:00:00+00:00

When I was about 12 years old, I was in school in Mumbai. At the school, I was in the Basketball team. I started playing the sport without any special reason; it was something to do. As I played the sport, I discovered that I was particularly good at some of the defensive positions (mainly because I was able to run faster than everyone else who was at the same school in my age group: an exceedingly small sample). This discovery was a revelation to me. I believed that the connection between “skill” and “success” was real: I believed that my coach would see that I had talent and pick me to play in regional matches. This didn’t happen; in fact, I remember being picked only once during my 5 years on the team. This is not a sob story. I wish I did feel bad about it, I wish I felt like I had been wronged, so that I could talk about recovering from that place. I don’t feel like that at all; I look back at that phase of my life as an amusing few years. This post is an exploration of my feelings about what happened then and how I make sense of it now.

In my school, there was a pecking order, like in every school and institution out there. In my basketball team, seniority was everything. The earlier you started practicing with the team, the better your chances of getting the coach’s attention. There were about 20 people who would attend most practices and were dedicated to the sport. In any basketball game, 5 people play on the court at any given time. Frequent substitutions were common for our team, and in actual matches which we played against other schools from the same city / district / state, the coach decided what substitutions to make. So, at the top of the pecking order, was the basketball coach, below it were some of the people who were in the “Under 16 (U-16)” age group and had been practicing for several years, then there were some of the junior people, like me, who had been a part of the team for only a couple years at that point, and finally, some people who had just joined the team and would likely drop out after a few weeks.

The practice schedule was relaxed. We practiced for about an hour each day. There were most of the typical parts of any basketball team’s practice sessions: Lay-ups, running, mock matches, three-point shoots and free-throw practice. The mock matches were the most exciting. I lived for those; they gave me the adrenaline rush that I craved: A real match, with a clock and real opponents (even if they were pretend opponents on your team).

Some of this was great fun. It was the first time I had friendships with people who were older than me, who had more “life experience” (whatever that means for a 12 year old). I remember one particular occasion when it was raining heavily and I had gone to practice, arriving completely drenched, only to learn that practice had been canceled. I don’t know why this particular instance stuck with me, perhaps because of my belief that I was a diligent member of the team.

Some of it was not so much fun: I was lightly bullied by some of the older people on the team who were not very good but had stuck on simply because of their seniority and dedication. I had a few run-ins with some of the other “favored” players on the team who were good at offense and generally pissed when I played some part in making them look like a fool on the court. The adult supervision was limited and I was tiny (about 150 cms, 40 kgs). I remember only a couple instances when the altercation became physical. I wasn’t damaged in any sense due to all of this. I am certainly proud that I didn’t give in to the bullies and continued to play the best that I could on the court.

I remember distinctly the coach’s partiality when picking people to play in actual matches. It was the first time I had run into anything of this sort. I was good at studies and that generally meant favorable attention from my teachers, and consequently, favorable outcomes when there were selections for things like the regional Mathematics Olympiad.

My run-ins with partiality were frustrating and I felt helpless. Several years later, I still remember this feeling well: Being convinced that the coach was making the wrong move when he picked someone to play, certain that I could do a better job. When we went to other schools to play in regional matches, the coach would pick the same people over-and-over, irrespective of how dismal their performance in the previous week/day/match.

This behavior was inexplicable; maybe he was just incompetent; maybe he chose the same people over-and-over because he believed that they gave him the best chance at winning; or maybe, he was in denial, convinced that his judgment was flawless despite all evidence to the contrary. At school, we called this kind of bias partiality. I have heard about similar experiences from most of my friends who went to school in India. I hear about this from my younger cousins who are in school now. We would talk about partiality when people were selected to be a part of the school’s quiz team, or to go to another city to participate in a debate. This list is endless. When I asked my parents why this kind of thing happened, they would tell me about the oft-repeated tenet: Life isn’t fair; that’s life. I accepted this principle, and embraced it, without knowing how to deal with the fallout.

We didn’t win many matches. The one time that I played in an actual match, I must have been on the court for about 10 minutes of the total 40 minutes, I don’t remember accomplishing anything match-changing. We lost that one too. Two things stood out, about this particular loss.

First, I wasn’t jubilant that I had been picked to play; I didn’t even tell my parents that I had been on the court, it felt like a trivial detail.(My parents suspected that I rarely got picked. They were content to know that I had had a good trip and didn’t press me for details. I am eternally grateful to them for that. I don’t think it would have done me any good if I had provided them the details, or if they had tried to console me, or asked the coach to pick me more often).

Second, that loss didn’t hit me any harder than the ones that I had watched from the sidelines. Every loss felt the same irrespective of whether I had been on the court. I was not able to write off the other losses as something I didn’t have much to do with, nor did I feel more responsible for the loss, when I was the one on the court. It was always a strange experience to get back on the bus and travel back to our school. Everyone was mostly sad about the match, but we were also happy about getting to go somewhere, experience the world alone, almost like adults (Turns out Michael Scott was right about bittersweet experiences).

This visceral experience with bias at a young age jaded me, I think. I found it easier to shrug off future losses and failures as just unfortunate strokes of luck. Today, looking back, I can see that I forged the path to where I am. I also like to think that I have tried my best to prevent and be aware of bias in my dealings with others.

P.S. I am writing about this after about a decade and if any of the “romanticizing the past instinctively” business is true, then the “good” parts that I talked about were decidedly worse than I portrayed them.

What's Great about Newsroom, the TV show?

2021-01-23T00:00:00+00:00

3 months ago, I had absolutely no idea that people found Sorkin’s writing annoying. Then, I heard that someone I knew had given up on watching Newsroom after watching the first few minutes of the Pilot episode. Hearing about that sent me for a brief whirl while I figured out what I liked about Sorkin and why. I think now I have a better understanding of that. I recently re-watched Sorkin’s Newsroom. Watching it again after a few years since the last time I watched it, I noticed that I was familiar with a lot more people in the show (BJ Novak from The Office, Clea DuVall and Sarah Sutherland from Veep). I also realized that the show is packed with information, nuance and characters who know what they are talking about. Why can you only either “love” the show or “dislike” it passionately?

People have talked about Newsroom endlessly. This positive and negative review of the TV show in The New Yorker cover all the bases. They are both talking about the same components of the show: the characters speaking articulately, with absolute confidence in what they are saying; each character being good at what they do and Sorkin’s ability to show the audience that this is how the system is supposed to work. The critics who don’t like the show find it unrealistic, annoying and heavy-handed. Let’s look at each of those adjectives in turn.

First off, let us do away with the “This is not how real people talk” critique. Sorkin is not going for realism; all of his work has been about people speaking in ways that are dramatized, that appear real but are very far from reality.

The reason I dismiss this critique out of hand is that I don’t believe TV shows or movies need to show reality as is. The shows and movies I like don’t reflect reality at all: Contagion (2011) was about a pandemic that ended abruptly with a vaccine; The Office (US) is a show about an office that continues to survive despite the disastrous decision making that goes on inside it; Veep (HBO) is about an incompetent vice president and her incompetent team reaching the highest levels of government; Homeland is about a CIA agent who engages in espionage and commits a series of increasingly questionable actions in pursuit of her goal. These shows do not aim to portray reality at all. In my mind, the demarcation is clear: Documentaries are about reality, TV shows and movies are dramatized versions of reality or an interpretation of what an “ideal” or “comically untenable” version of reality would look like. If the critics who are negative about Newsroom are doing it because it’s not “real” enough, how would they rate any of these other shows? How would they critique anything other than a documentary and fantasy? Here’s my question: Is a TV show’s portrayal of events and their closeness to reality important to you?

What about the critics who say that this kind of writing is “annoying”? I believe I understand the reason that people find it annoying. One of the supporting characters on Newsroom, Sloan Sabbith (played by Lisa Munn) is a great example of a character that people find annoying. She is an extremely well-educated economist with a Post Doctorate, who has a ton of experience and knows exactly what she’s talking about every single time she talks about anything technical. She’s also “likable” (in a sense) because she is socially inept, aloof and talks about not having a lot of female friends. In a lot of her scenes, she is talking about a journalistic ideal or an economic principle. Her explanations are simplified and reductive, but they get the point across. She talks about Glass-Steagall, the Japanese radiation level announcements in March 2011, the subprime mortgage crisis that brought about the 2008 meltdown, finding out more about a deal she knows is happening soon and is intrigued by but doesn’t know the details about, and about her new “Bloomberg Terminal”, which she uses to figure out which network a potential buyer is going to buy. In all these scenes, her dialogue is dense, articulate, and filled with important information. There’s no filler, she doesn’t ruminate or explore what she is about to say before coming to a conclusion. She’s there and impatiently waiting for the listener (and the viewer) to catch up. I find this kind of story-telling challenging and engaging. This is the major reason that I like Sorkin: He turns TV, a traditionally “passive” medium, into an “active” one by forcing the viewer to keep up or risk missing the nuance in each sentence.

I have thought about the “active” vs. “passive” medium dichotomy for a while now. After a full day at work, I rarely used to want to read or write anything. Lately, I have been pushing myself to read and write after work and reduce the amount of time I spend watching TV shows. This is because I want to ensure that I am spending a big chunk of my time with active mediums, mediums that require me to think and stay engaged, rather than passive mediums, like TV shows or online videos, which I can turn on, tune out and let my mind wander aimlessly. Admittedly, passive media is great in some cases (e.g. plane rides). On a daily basis though, passive mediums are best avoided.

Finally, what about the critics who say that this kind of writing / story telling / TV show making is “heavy-handed”? There are several ways of making this point; one of the common tropes is to say that Sorkin’s characters are not the “Average Joe”. They are incredible; they are at the top of their class, they have had the best education / experience / relationships, they are confident about who they are and what they want, they are confident that they want to remake the world and they are sure about how they will do it, they play instruments / are talented / good with people / put other people at ease. Another common method used to reach the same endpoint is to say that Sorkin fetishizes intelligence: His characters are intelligent, way more than a “common” person; they have the capability to take in a huge amount of information, such as books or dense reports with 1000s of pages and retain almost all their life’s memories, remembering everything that happened several years ago in great detail.

Clearly, there’s no doubt that Sorkin does this: another supporting character on the Newsroom, Jim Harper (played by John Gallagher, Jr.) is good at his job, freaky good. He has spent time with the US Army as a part of his reporting, he is very good at being Mackenzie’s senior producer in the studio. His experience is revered and his colleagues hold him in high regard. He plays the guitar, and is good at getting people to talk to him. He takes stands when he feels like he has to, and never backs down, ending with the loss of press bus privileges during a campaign and his relationship. None of this is abrasive to me; in fact, I absolutely want to see Jim’s point of view, not least because I believe that I would agree with him on a lot of things. We see this in concrete terms when his girlfriend, Hallie, gets a job at a news site called “Carnivore”. Jim notices that Hallie’s contract mentions that her bonuses are tied to page views on her stories. Jim believes that this is detrimental to journalism and incentivizes Hallie to write something that she thinks will get views, rather than something she thinks is important and the public should know about. Jim’s uprightness about this point is painful, as he lets his relationship with Hallie deteriorate over an ideological difference. It’s important to point out that even though I agree with Jim on his point here, I am certain that I wouldn’t let it get in the way that he did. There is a vicarious living element to relating with characters who don’t back down.

I don’t think the “heavy-handed” approach is a good critique. “The show is insufferable because it’s filled with characters who know too much and speak too elegantly”? That argument is flimsy to me. Exceptional TV shows are not filled with average people. Average TV shows are filled with average people. Seinfeld is not exceptional because the four main characters are run-of-the-mill; Jon Stewart’s Daily Show was an exceptional piece of comedy television, not because Jon Stewart and the correspondents (Colbert, Helms, Carell, Samantha Bee) were average; Nathan for You is a show that is filled with average people, seen from Nathan’s point of view, someone who is trying to figure out the lengths to which people will go, when they are on TV and not really sure about the show’s goals, a decidedly uncommon thing to think about or try to televise. Expecting great television that is filled with average characters is a fallacy. The people making these “intelligent” shows, the characters they are writing, and the actors who are playing these characters: none of them are average, that is a necessity if they want to continue to make the kind of TV that they do.

My advice? Watch the show and decide for yourself. It is likely that you will like the show. It is equally likely that you won’t like the show. Once you have made up your mind about which camp you are in, send the two New Yorker reviews to someone who hasn’t watched the show and ask them to watch the show.

New Yorker Magazine Subscriptions

2021-01-12T00:00:00+00:00

About 11 months ago, in Feb 2020, I signed up for a 12 week subscription to the New Yorker magazine. It was the first print magazine I came across that supported International shipping and would ship to Japan. It was also a magazine that I had heard about a lot on TV shows and podcasts (Seinfeld happens in New York, and in one episode, the main characters discuss a New Yorker cartoon in depth; This American Life and The Daily are both podcasts that are made by crews that live in New York) After 24 weeks, I ended up canceling my subscription because I couldn’t keep up with the sheer volume of content that was being delivered each week. This post is a recap of my experience.

I started by reading several articles on the website. Most New Yorker articles were available and they were good; they were articles about a specific issue or a book or someone’s experience which was pointed, had a fixed story arc and had a clear beginning, middle and end. I liked this kind of writing and I was ready to spend more time reading it (back then, I was still using Twitter regularly). The magazine’s subscription had an ongoing sale plan and I would have been able to get 12 weeks of magazines for about $12, which felt like a steal and I bought the subscription. My main concern when I bought the subscription was that it would be too much to read and I would simply never get through it and cancel the subscription after some time because I felt bad about not reading the magazines. This is exactly what ended up happening.

Why I signed-up for a subscription

Over the first 4-6 weeks, from Feb 2020 to the end of Mar 2020, I was barely keeping up. I was spending about 2-3 hours on both days of the Weekend reading the magazine. I came across some great articles and it was a pretty soothing experience to be able to read something that was not closely connected to current affairs or news or anything that was happening right now. The magazine’s coverage was also very wide, and I got a kick out of reading the “Around Town” section which had short snippets about live concerts, Broadway shows, other events happening in New York.

I live in Tokyo and for the first time, I felt that all the talk about New York being the cultural center for theater and jazz was not just talk. For the first time during my time here, I questioned my belief that culture could not be geographically contained. After moving to Tokyo, I had gone to several rock shows and classical music concerts and piano recitals, for the first time in my life. They had all been amazing experiences; but I did not kid myself. I knew that Jeff Daniels was not going to star in an Aaron Sorkin production of To Kill A Mockingbird that would be performed in Japan. The audience was not big enough; the content was not related enough.

The magazine also had a delightful fiction section and several book reviews, a TV show and movie review which lead me to some great books and movies. The final part of the magazine were the long form articles. These were generally on some solid topic which was not related to anything that was going on at that particular moment of time. It was just a topic that came out of nowhere. These also often included “Profiles” of people; these profiles were very interesting. I picked up Sapiens (Harari) after reading the author’s profile (albeit I didn’t end up liking the book).

Finally, the cartoons. Oh, a lot has been said about New Yorker cartoons. They are witty and involved and about 3 times out of 10 they went over my head because I didn’t understand the reference. I felt pretty-much like Elaine in The Cartoon (Seinfeld, 1998). Apart from the single panel cartoons, there were also some elaborate drawings in the “Sketchpad” section and these were my favorite part of the magazine. (Personal favorites: Hammurabi’s Code of Manners and A Psychiatrist and an Artist Walk Into a New York Bar).

Why I canceled my subscription

I couldn’t keep up with the sheer amount of content in the magazine. There are 46 issues every year. Every issue is packed with a lot of amazing articles that I didn’t want to miss: This made skipping over articles or saving them for later hard, I knew that once I moved on from one particular issue the probability that I would come back to read an article in that issue was very small. (For me, somewhere around 0.001)

Apart from this magazine, I was reading non-fiction in English and I was also reading several Japanese news articles every day to get better at the language. I was not going to give up my non-fiction reading or my Japanese learning to stick with the magazine; the point of reading the magazine for me was to have something to read during the times that I didn’t want to read the news or a blog or something technical: Something to read, instead of watching a TV show or YouTube.

Several months later, I do believe this was the right decision. I have ramped-up both non-fiction reading and Japanese learning in the time since. I do feel like I might have been able to work something out if I had been a little bit more motivated to read and engage with the magazine in a different, gentler manner; but I don’t know if that’s true or if I am simply idealizing my desire to read a magazine and not giving up on something that will always remain a pipe dream.

This week, as I started thinking about this again, I searched `how do people have time to read the new yorker` and came across this article:

If anything, an article in the New Yorker is more like a pipe or a cigar. It’s consumed, not with the fidgety and neurotic posture of an insatiable itch-scratcher, but with the deliberate focus of a Tibetan monk. And unlike smoke, these are stories that last. … I’m glad I didn’t miss it, but I’m still able to rest easy — even though I’m certain I’ve missed countless stories that are just as good.

In the end, I figure it’s a good thing we don’t all know the same stories, anyway. If we did, what would there be to talk about when we finally met?)

Now, I have pinned the New Yorker’s latest magazine’s digital edition on my home computer’s browser. I plan to start by reading the Fiction section and some of the long form articles that I am interested in. Eventually, I want to read some of the articles that I am not really interested in to see if they involve discussion about a topic that I find interesting after reading the article. For the reader of this post, I strongly recommend checking out the Digital edition of the magazine here.

Reserve Bank of India's Interventions to maintain excess liquidity - Part 2

2021-01-11T00:00:00+00:00

In the previous post, we looked at what the RBI has been doing over the 2020 calendar year. The over-all effect of RBI’s interventions have been that the exchange rate (INR to USD ratio) has remained within a narrow window, despite the unprecedented capital inflows into the Indian equity market. This is by design, and the RBI has tried to explain their policy a few times in various documents. Although this excess liquidity can theoretically cause high inflation and have an adverse impact on the currency, a couple other things are happening simultaneously and making this policy prudent (existing over-valuation and RBI’s strong balance sheet). After reading some of these documents, here’s my thesis of the advantages and disadvantages of what the RBI has been doing, as I best understand it.

Advantages

Monetary Policy Transmission

One of RBI’s goals is to set monetary policy. In an ideal world, they would set a policy with a given policy goal (e.g. bring down food inflation) and this policy would be transmitted to end-consumers perfectly through the various economic institutions that sit in between them and the RBI (e.g. state banks, corporate investors, foreign investors, etc). As you might have guessed, we don’t live in an ideal world and the RBI has to have a policy to ensure that their monetary policy is transmitted to end-consumers and has the required effects.

By maintaining excess liquidity in the system, they are trying to ensure that policy transmission is close to perfect by maintaining the operating rate (the interest rate at which banks borrow in reality) close to the policy rate (the interest rate at which banks borrow directly from the RBI). The high supply of local currency and constant deregulation at the RBI ensures that banks can borrow at rates close to the policy rates. This can be seen in the release from 8th January 2021: The short term market repo rate was 3.20%, whereas the policy repo rate for this period was 4.00%, and the reverse repo policy rate was 3.35%.

This can be a crucial tool during a recession to increase consumer spending and economic activity, as it has been found to be effective at (say) increasing the amount of real estate loans that are being issued and helping firms and households start spending again. The RBI has noted this in their last Monetary Policy Committee (MPC) meeting in early Dec 2020:

Point 33:

My own view is that just as the reversal of the liquidity drought had led to a revival of growth that was visible in the high frequency data for February 2020, the various measures to make liquidity available through the economy not only helped firms survive but have also revived demand. The bank credit growth figures show a turnaround but underestimate it. … Real estate inventory is beginning to move. Both households and firms have deleveraged, are cash rich, and ready to spend. There are early signs of firms beginning to invest.

– https://www.rbi.org.in/Scripts/BS_PressReleaseDisplay.aspx?prid=50831

Control over-valuation of the INR

When foreign inflows into an economy are high due to a perceived increase in forward earnings, the risk of over-valuation of the local currency starts going up. This over-valuation adversely impacts exports (As the local currency appreciates against reserve currencies, foreign buyers will have to pay more for the same commodity, provided that their local currency is not appreciating as quickly). To curb over-valuation, the Central bank will purchase the foreign inflows and park them as Forex reserves on their balance sheet.

This absorption of the capital inflow by the central bank is also lucrative as it ensures that a sudden-stop in the capital inflows will not destabilize the market. The central bank can start removing liquidity from the market gradually, allowing the sudden-stop gradually. In India’s case, the probability of a sudden-stop is low as 1-year forward earnings are still high, and there are no big changes in policy or government that are expected until Dec 2021. The next central government election is only in 2024, a long time away.

Once again, here’s an excerpt from the MPC minutes:

The intervention that is raising foreign exchange reserves is required because over-valuation of the rupee can hurt exports, raise country risk and lead to a sharp depreciation later. Prolonged inflows can lead to over-valuation without intervention. Surges and sudden stops of capital flows to emerging markets due to advanced economy quantitative easing have hurt emerging market growth in the decade after the global financial crisis.

Disadvantages

The RBI has been weighing these disadvantages against the economic growth that is being supported by the excess liquidity. Ultimately, their decision to stick with their current policy comes down to their belief that these disadvantages are not devastating even in the worst case and that the risk is bearable.

Impact on Consumer Price Index (CPI) Inflation

Excess liquidity’s biggest impact is on CPI inflation: the excess liquidity in the system makes it easier for firms to invest and they subsequently raise their commodity prices. In some cases, it is possible to maintain high liquidity but prevent an increase in inflation due to this. These cases generally happen when the inflation rates are already quite high and expected to remain high due to external reasons. As we saw in the previous post, the current high inflation rates in India (October CPI inflation = 7.6%) are caused by the COVID-19 pandemic’s impact on supply chains and some untimely rains in the harvest season. RBI’s monetary policy is not connected to the current high inflation rate.

Even in these situations, the already-high inflation rates can be exasperated by the liquidity surplus. So, the effect on inflation should be closely watched and liquidity suppression tools should be used when the trend shifts. RBI is on track to do this, although some economists and analysts believe that RBI’s hand will be forced in 2021 as (say) food inflation starts to go up again before or during the summer months.

Analysts suggest the RBI will be forced to address the [liquidity] glut early in 2021.

Here’s an excerpt from the MPC minutes:

To the extent it is transient the contribution of excess liquidity to cost push inflation is limited. In an open economy import competition also caps price rise, especially with a rupee that is tending to appreciate, provided tariffs and taxes are moderated.

RBI keeps government borrowing costs low

As the economy is stressed due to various state-wide lockdowns and a sluggish consumer, the government has been borrowing heavily. The budget deficit for FY2020-21 is now estimated to end up at about 8%, which is twice the budgeted value of 4%. This deficit is being made up by government debt issues. A lot of these debt issues have been purchased by the RBI. This has ensured that the government’s borrowing costs in the long term (i.e. 3-, 5- and 10-year bond yields) have actually gone down.

This is one facet of the RBI’s theory of Economic Capital Framework. This theory enhances the central bank’s role from simply avoiding financial volatility, to being a backstop when the government’s finances are deteriorating. The central bank aims to achieve this goal by maintaining a strong balance sheet and transferring balance sheet capital surplus (equity above the required rate and whole of net income) to the central government. (Note: I have been reading about this theory and I am still trying to understand it’s background and where it came from. I might do a series of posts about this in the future.)

Short-term rates crash

This article makes an interesting claim:

Lower shorter rates without a similar drop in long-term borrowing costs means a steeper yield curve, which tends to undermine efforts to stoke growth.

This is also discussed vaguely in the MPC minutes:

Point 40.

As long as the MPC stance is accommodative durable liquidity will be in surplus and short-term rates will not rise above the reverse repo rate. Rates have fallen below the reverse repo because of the combination of excess foreign inflows, intervention and reverse repo access limited only to banks. Even so, excess liquidity is still absorbed. Regulatory exposure norms can help prevent excess low rates driven short-term borrowing that creates risks.

Point 46.

I believed then and believe now that this reduction of rates carries significant risks and very little rewards. The rewards are low because long rates are what are relevant for stimulating investments and supporting an economic recovery; a steepening of the yield curve by a reduction in short rates does not accomplish this. Also, a reduction in long rates that stimulates investment not only increases demand in the short run, but it also stimulates supply in the medium term as the new capacity becomes operational, and this new supply dampens inflationary pressures.

I don’t know whether this short-term interest rate crash can have long-term impacts. It looks like the Monetary policy committee is divided on this point and while the decrease in short term rates is not being supported by any of the members and is being opposed by some members, they don’t feel that the risk has risen to a level where they have to act and intervene to support this rate.

Reserve Bank of India's Interventions to maintain excess liquidity - Part 1

2021-01-11T00:00:00+00:00

In December 2020, I came across several articles about India’s Central Bank, the Reserve Bank of India, intervening in the open market to maintain excess liquidity of the Indian Rupee in the market. With my elementary understanding of currencies, I understood that this excess liquidity of INR in the market, would keep the currency weak against other currencies, and this seemed undesirable. In this post and the next one, I will delve into the complicated reasoning behind the RBI’s choices and their delicate balancing act which seems to be associated with a high but manageable amount of risk.

Premise

Last month, I came across several articles that were talking about the same kind of topics:

Huge outflows from India’s bond markets (bloomberg.com, theprint.in)
RBI’s aggresive buying of the USD flowing into Indian equity markets (bloomberg.com (1), bloomberg.com (2), moneycontrol.com)
RBI’s growing policy woes and their potential lose of control of the INR (Indian Rupee) (msn.com, bloomberg.com)
INR’s momentary appreciation, after the addition of India to a Currency manipulators watchlist by the US, which lead the RBI to reduce it’s aggressive buying (finance.yahoo.com)

These topics are all closely connected to each other. In fact, they are all so closely connected that reading these articles a couple times left me very confused about what was going on, what the RBI was doing and why the RBI was intervening into a market that wanted to increase the value of the INR. I will cover the first two points in this post; and a future post will cover the last one.

What’s going on? (before RBI interventions)

There are a few inter-connected things that are happening simultaneously. (This is a subset of all the things that are happening in the economy)

Significant drop in the Inflation-adjusted 10-year Indian Government bond yields

Over the 9-month period from Jan-Sep 2020, outflows from the bond markets totaled up to $14bn. In comparison, the investment in China’s bond market amounted to about $120bn. This exodus could be fueled by the low levels of inflation-adjusted bond yields: The bond yield hovers around 6%, whereas the inflation number is hovering around 6.8% to 7.6%. While this inflation-adjusted number might not deter domestic investors, for foreign investors, this negative inflation-adjusted bond yield number makes it hard to justify investment in the Indian bond market. Although, it looks like the outlook for 2021 is optimistic:

“We are constructive on the Indian market next year as India is a high beta market in EM,” said Emily Alejos, chief investment officer at Cartica Management in Washington D.C. “We expect all EM to prosper.” – msn.com

Another thing to look forward to is the inclusion of the Indian 10-year benchmark bond in global indexes, which will automatically bring inflows through Index funds.

Huge capital inflows into the Indian equities market

On the flip-side, capital has flown into the Indian equities market. For the past, this can be seen in the increase of the main index, the NIFTY50 which went from 12,182 points to 13,981 points, an increase of 14.76%. One-year forward earnings are about 28 times according to the msn.com article, and this has pushed cumulative Foreign buying of Indian equities to about $20bn, from 1st Jan to 15th Dec 2020. This is the highest since 2012 and has caused an unprecedented build-up of USD with state banks.

The outlook for this continues to be very strong with most analysts expecting this rally in the investment into Indian equities to hold atleast until Dec 2021. The Indian economy’s recovery has been sluggish so far, but with the widespread inoculation of vaccines and a reboot of economic activity and engagement to 90+% of pre-Covid levels, foreign investment might even go up further.

(As we will see in the second post of this series, this increase in foreign inflows causes a risk of “Over-valuation” of the rupee and RBI has been fighting back using aggresive USD purchases)

Supply-side shocks raising all Inflation numbers (caused by COVID-19)

As the COVID-19 pandemic started spreading in India, the government acted swiftly (to a fault and perhaps prematurely) and imposed a nationwide lockdown. This lead to a breakdown of most supply chains as long-distance trains stopped operations and all state borders were effectively closed. There was a push to ensure that the supply of essential goods and truckers would remain unaffected, but this didn’t work well enough and the bureaucratic system to grant permits to truckers most probably took more time than the government initially anticipated.

This sudden supply-side shock had an expected impact on the CPI Inflation across the country. Food inflation started shooting up from April, but the expectation of a good harvest kept fears low initially. As October rolled around, untimely rains in Maharashtra (a major farming state) lead to the food inflation going up to 11.07%, an all-time high for this number. Overall inflation also kept going up, and ended up at around 7.6% by early December.

The key finding here is that this inflation is not connected to any of RBI’s monetary policies. In fact, according RBI’s assessment, they confidently claim that the effect of their currency interventions on inflation have been minimal:

Statement by Dr. Ashima Goyal

Point 43:

To the extent it is transient the contribution of excess liquidity to cost push inflation is limited. In an open economy import competition also caps price rise, especially with a rupee that is tending to appreciate, provided tariffs and taxes are moderated.

– https://www.rbi.org.in/Scripts/BS_PressReleaseDisplay.aspx?prid=50831

Note: For this series, I will take the RBI Monetary Policy Committee members at face value. In this regard, healthy skepticism is appropriate, and I will write more about this if I go out to find something critical about the RBI’s monetary policy.

Rupee’s weak recovery from the initial recession (caused by COVID-19)

Once again, moneycontrol.com and the INR/USD spot rate chart both show that the exchange rate was kept in a tight window up to July; the INR/USD ratio stayed above 74.7 from 20th Mar to 21st Aug 2020. The currency’s appreciation was allowed for a short period of 10 days from 21st Aug to 31st Aug, when it dipped to 73.12. After this short hiatus, the RBI restarted it’s intervention to support the INR/USD ratio at or above 73.5, and not allowing the INR to appreciate below this number (Note: A decrease in the INR/USD ratio is considered an INR appreciation).

Across all Asian currencies, INR’s initial depreciation was the highest. After September, INR’s appreciation was the smallest as shown in this chart. One should note that the prevention of appreciation of the INR after Sept, and the prevention of momentary depreciation after some border skirmishes were both direct results of RBI’s intervention. This intervention was egregious enough to get India added back to a Currency manipulation watch-list by the US.

(As India waits to be added to global bond indexes, one of RBI’s goal seems to have been to curtail volatility in the exchange rate by keeping it within a tight window. India’s bonds have not been included in global indexes mainly due to an overly restrictive regulatory framework, which has not been modernized quickly enough)

What is the RBI up-to?

This graph of the RBI’s foreign exchange reserves tells the whole story. IT shows the foreign exchange reserves going up from $450bn at the beginning of 2020 to almost $580bn at the end of 2020.

The actual numbers can also be found in the RBI’s Weekly Statistical Supplement

Sidenote: This page on the RBI website has become my favorite page to get macro information about RBI’s activities. RBI also maintains this, the Database on Indian Economy, but I have not been able to figure out how to download an spreadsheet of time series data or use the native “Filter” feature yet.

Date	Value	Year-on-Year increase	Supplement
2018-01-05	385103.9	14.415901	2017, 2018
2019-01-04	368077.2	-4.4213263	Link
2020-01-03	424936	15.447520	Link
2021-01-01	537474	26.483518	Link

(Value is the amount of Foreign exchange reserves held by RBI in millions of US$)

The 26% increase in reserves from the previous year is stunning. This has also pushed RBI to the fifth place in the list of central banks with the largest reserves of USD.

This purchase of USD, offsets the increase in the capital inflows. The inflows into the Indian equity market were $20bn, whereas RBI has purchased nearly $113bn in USD in the same amount of time. This asymmetric intervention has helped them flood the market with excess INR liquidity and effectively prevent the INR from appreciating against the dollar. It was only after RBI was added back to a currency manipulators watchlist did the INR appreciate in real terms. This appreciation was short-lived and the RBI got right back into USD purchases. They continue to make weekly purchases and increase their reserve foreign exchange reserves. Why is the RBI involved in preventing the appreciation of the Indian Rupee? I will try to present a few advantages and disadvantages of the RBI strategy in the next blog post (to be released tomorrow, 2021-01-11).

Comparison of metrics of COVID-19 vaccines

2021-01-09T00:00:00+00:00

Over the past few days, several countries have approved vaccine candidates for emergency use and even wider use. In the US and in India, 2 vaccines have been approved. In the UK, both vaccines that were approved in the US have been approved, along with one that has been approved in India (under a different brand name). After seeing all these approvals, I wanted to understand what their comparative merits and demerits were. In particular, I wanted to understand the size of their clinical trials, the confidence each had in their efficacy numbers and their safety profiles. To this end, I read the “Primary results” research papers for all 4 vaccines. This post will try to compare the headline numbers, and explain some of the similarities between the trials, and the things that seem to have happened only with a single vaccine.

Side note: These papers were clearly written with very little medical terminology. I was able to understand them once I knew what terms like Seroconversion and Immunogenicity meant. The papers do have a considerable amount of graphs and numbers, so parsing them is an important part of reading them.

Here’s a quick comparison table of the efficacy numbers, along with their 95% confidence intervals.

Last updated: 11th July, 2021

Vaccine	Approved countries	Trial size	Efficacy	Confidence intervals
Oxford-AstraZeneca¹ (2 standard doses)	UK, India (as Covishield)	8800, 1:1 vaccine:placebo	62.1%	34.7 (41, 75.7)
Pfizer	US, UK	42,000, 1:1 vaccine:placebo	95%	7.3 (90.3, 97.6)
Moderna	US, UK	30,400, 1:1 vaccine:placebo	94%	7.5 (89.3, 96.8)
Bharat Biotech (Covaxin)²	India	25,800, 1:1 vaccine:placebo	77.8%	21.2 (65.2, 86.4)

From just these numbers, it is abundantly clear that the Pfizer and Moderna vaccines are way ahead of the rest of the vaccines in this pack. The timing of the approvals in India is also worth noting: Both the approvals came days after the approval in the UK.

Another astonishing fact is that the Bharat Biotech vaccine which has been approved by the regulator in India does not have a valid efficacy number yet because their Phase 3 clinical trial is still ongoing, and not even the Interim results from those trials have been released. The regulator has claimed that it is a safe vaccine, even though the largest trial for that vaccine has been the phase 2 trial which had 380 participants.

Astra-Zeneca’s Clinical Trial

As I was reading the paper with the interim results for the AstraZeneca, I noticed that it was markedly different from the other papers in some major aspects.

Inaccurate measurement leads to the creation of 2 sub-groups

In the trial COV0002 that was held in UK, there seems to have been some lapse between the manufacturing of the vaccine and it’s quality check. Here’s the relevant part of the paper:

Initial dosing in COV002 was with a batch manufactured at a contract manufacturing organisation using chromatographic purification. During quality control of this second batch, differences were observed between the quantification methods (spectrophotometry and quantitative PCR [qPCR]) prioritised by different manufacturing sites. In consultation with the national regulator (Medicines and Healthcare products Regulatory Agency), we selected a dose of 5 × 1010 viral particles by spectrophotometer (2·2 × 1010 viral particles by qPCR), in order to be consistent with the use of spectrophotometry in the phase 1 study (COV001), 5 and to ensure the dose was within a safe and immunogenic range according to measurements by both methods. A lower-than-anticipated reactogenicity profile was noted in the trial, and unexpected interference of an excipient with the spectrophotometry assay was identified. After review and approval by the regulator, it was concluded that the qPCR (low-dose) reading was more accurate and further doses were adjusted to the standard dose (5 × 1010 viral particles) using a qPCR assay. The protocol was amended on June 5, 2020, resulting in enrolment of two distinct groups with different dosing regimens with no pause in enrollment

This inadvertent creation of a new sub-group in the test, with an already small size of 3744 people lead to the reduction of the people who got the “2 Standard Doses” regiment to 2377. The remaining 1367 people were the ones who were included in the “Lower Dose/Standard Dose (LD/SD)” regiment and the only thing that was required to become a part of the group was that you would have had to arrive at the trial site between 31st May 2020 and 10th Jun 2020. The careful composition of the original trial group was affected due to this sub-group, and the observed higher efficacy within this group (90% in LD/SD against 60.3% in SD/SD) could be a consequence of this lack of proper composition of the subgroup. The authors say that although this number isn’t conclusive by itself, it’s presence indicates that conducting a new trial with a smaller primer dose (the first dose) will be useful.

Pooling of data across 3 countries

AstraZeneca’s phase 3 Interim results are actually the results of 4 trials that are being conducted simultaneously. Of the 4 trials, 2 are happening in the UK, 1 in Brazil and 1 in South Africa. The size of each of the trials is small and ranges from 1077 (COV1 in UK) to 3744 (COV2 in UK). Of these 4, only 2 had enough cases to reach the Interim results announcement stage.

Wide confidence intervals

This is perhaps one thing that one should take note of. The lower bound of the confidence interval for efficacy is 41%. While there’s no telling what the actual efficacy is, this wide confidence interval raises some questions about the size of the trial, and whether the size should have been bigger to ensure that the interval was smaller.

The author’s also note this about the 90% efficacy that is seen in the LD/SD subgroup. The CIs for this value is 67.4% to 97%, a width of about 29.6%.

Use of a low dose for priming could provide substantially more vaccine for distribution at a time of constrained supply, and these data imply that this would not compromise protection. While a vaccine that could prevent COVID-19 would have a substantial public health benefit, prevention of asymptomatic infection could reduce viral transmission and protect those with underlying health conditions who do not respond to vaccination, those who cannot be vaccinated for health reasons, and those who will not or cannot access a vaccine, providing wider benefit for society. However, the wide CIs around our estimates show that further data are needed to confirm these preliminary findings, which will be done in future analyses of the data accruing in these ongoing trials.

Vaccine safety

The safety of each vaccine was also a major endpoint in all these studies and they had several categories of adverse events. Directly related to the vaccine, the adverse events were categorized as “Local” and “Systemic”, and these were solicited for 7 days after the first and second injection. The thing to look for here is a higher incidence of local and systemic adverse events in the participants who are in the Vaccine group.

Apart from this, throughout the study, the trial participants were supposed to report adverse events and severe adverse events to the trial conductor. Here, the thing to look for is similar incidence of serious adverse events in both the vaccine and the control groups. (These included things like death by homicide in the AstraZeneca trial or death by suicide or cardiac arrest in the Moderna trial; they are used to record the adverse events that happen during the trial but are not related to the vaccine or the trial’s characteristics)

I noticed that 2 things happened in all three phase 3 trials:

Younger participants (16-55) were more prone to local and systemic adverse events than older adults (>=55). The most common adverse event was pain at the injection site after the first and second dose, and these numbers hovered around 5-10% of all the participants. Fortunately, the resolution time for these adverse events was within 3-4 days in all trials
The vaccines didn’t lead to “enhanced respiratory disease after infection” (Moderna). This was a concern that was voiced by several scientists and journalists who are familiar with this issue when the vaccine development was starting. The Moderna paper explained this well:

The mRNA-1273 vaccine did not show evi- dence in the short term of enhanced respiratory disease after infection, a concern that emerged from animal models used in evaluating some SARS and Middle East respiratory syndrome (MERS) vaccine constructs. 23-25 A hallmark of enhanced respiratory disease is a Th2-skewed 12 immune response and eosinophilic pulmonary infiltration on histopathological examination. Of note, preclinical testing of mRNA-1273 and other SARS-CoV-2 vaccines in advanced clinical evaluation has shown a Th1-skewed vaccine re- sponse and no pathologic lung infiltrates.

The Bharat Biotech vaccine paper also mentions this as one of the points that makes the vaccine a promising candidate worth investigating and safe enough to move into the phase 3 trial stage.

Efficacy calculation

Here’s the calculation of the actual efficacy percentage:

Efficacy = 1 - (IRR_vaccine / IRR_control)

where IRR_vaccine = Confirmed cases in the Vaccine group / Person-years of surveillance and IRR_control = Confirmed cases in the Control group / Person-years of surveillance

“Surveillance” is the amount of time that the trial participants were followed around for, from a given number of days after the second dose, until the date of the data-lock. This number of days after the second dose varies depending on the vaccine, and it was 14 days for Moderna and AstraZeneca, whereas it was only 7 days for the Pfizer vaccine.

The efficacy is often calculated as the “primary endpoint” of these clinical trials, and includes all the participants who tested negative for COVID19 on Day 0 (i.e. the day that they were randomized and given the first injection). For a participant to be considered negative, they need to test negative on the Serology test (i.e. Test for antibodies to the virus) and the Virology test (i.e. Test for the virus, NAAT or RT-PCR). This definition is slightly different in each study, and the numbers for the complete group, including those who were seropositive at the start of the trial are also reported.

Link to papers

All these papers are available under OpenAccess from the journal websites. All the papers linked here are from the Phase 3 clinical trials.

Vaccine	Paper (Phase 3 results)	NYT Vaccine Tracker
Oxford-AstraZeneca (2 standard doses)	Voyset 2021	AstraZeneca
Pfizer	Polack 2020	Pfizer-BioNTech
Moderna	Baden 2020	Moderna
Bharat Biotech (Covaxin)	Ella 2021³	Bharat Biotech

Only Interim results of the phase 3 clinical trial have been published. For the AstraZeneca vaccine, I am reporting the 2 Standard Doses efficacy. This trial had some peculiar mixing of a half-dose and a standard dose variant which had an efficiency of 70.4% (54.8, 80.6), but this is not the variant that will be inoculated to people. In fact, this (half-dose, standard dose) variant seems to have come out of an inaccurate measuring methodology, and was not a planned variant. More on this later. ↩
Updated 11th July, 2021. Phase 3 trial results were published on the preprint server medrxiv.org on 2nd July, 2021. These results were not peer reviewed when this post was last updated. ↩
Phase 2 results: Ella 2020 ↩

Unionized Software Engineers - Is the Social Cost spread out disproportionately?

2021-01-06T00:00:00+00:00

We're Alphabet workers. We’ve been organizing for over a year, & we’re finally ready to share why.

This morning, we're announcing #AWU, the first union open to *all* workers at any Alphabet company.

Every worker deserves a union—including tech workers.https://t.co/m2Qmjwz32V
— Alphabet Workers Union (@AlphabetWorkers) January 4, 2021

Today, there was an article announcing a new workers union for all employees working at Alphabet, Inc., Google’s parent company, the Alphabet Workers Union. The union is constituted by about 400 employees, an extremely small percentage of the total workforce of nearly 250,000 people. The union dues are 1% of total compensation.

We still have discrimination, harassment, & retaliation at Google. Over half our workforce are TVCs—paid less, w/fewer benefits, for often the exact same work.

…

We deserve meaningful control over the projects we work on & the direction of this company.

Here’s where it gets interesting: The union is a minority union. As the union doesn’t have a majority of all the employees as members, there is no basis for them to walk up to Alphabet’s management and demand to negotiate a Union contract. Instead, they are going to use their union as a more abstract entity that will be used for other purposes. The question that came up when I was discussing this with a few co-workers was whether the social cost of this particular type of union is spread out disproportionately between the employer and the union members?

I have been reading up about unions after reading The Box, the book about the introduction of the shipping container and how it took over the world. The most fascinating part of the book for me, was the explanation of the “transition” period: the period when shipping companies, ports, and shipping users were deciding whether to move from break-bulk shipping to container shipping. Break-bulk shipping involves the packing and unpacking of material at each intermediate port. These packing, unpacking, re-packing steps were carried out by dockworkers. These dockworkers were a part of unions across the world. In some countries, they embraced the shipping container, deciding to improve their skills and be a part of the crew that would help initiate the transition. In other countries, dockworkers unions where opposed to the idea and did absolutely everything they could to ensure that the shipping container would not be introduced at the port that they were working at.

By “absolutely everything”, I mean a set of general strikes, restrictive union agreements, and increases in wages. One of the most fascinating clauses that one union agreement had was a clause that each hatch on a ship would have to be manned by a minimum number of people, irrespective of the actual amount of cargo that will be moved through that hatch. This lead to cases were the minimum was 16 people, but only 8 people would be working at any given time. To me, this indicated a broken system that was not ready to adopt new technology. Was my assessment fair? No, it was wrong.

Organizing to collectively bargain is a major part of the reason that a lot of workers have more leverage over their employers actions. Unions also have power in political elections, which gives them more attention from policy makers, and enables them to push for their policy preferences. This system has it’s advantages and can not be dismissed out-of-hand. This post will explore what is really going on here, look at the issue from a few perspectives, and try to untangle the various threads.

From society’s perspective, the negotiation between a union and the employer is a near-zero-sum game: Union membership has some major costs, mainly, union dues that one must pay constantly, the possibility of strikes when the union as a whole or the union leadership is unhappy with the work conditions, and the possibility of loss-of-pay during the strike while union members are paid out a lower stipend or given rations, using the fund maintained using their union dues.

When the union is happy, the cost is almost zero for both employee and employer. There is the union dues cost, which is balanced out by the time that the employer spends to ensure that their policies are not exploitative and will not anger the union enough to cause a strike.

When the union is unhappy, for a traditional union, the cost is still evenly spread out: The company loses their revenue, their ability to fill orders and a major drop in production, which will affect their yearly output targets. Union members bear an equal cost, as they are essentially out of a job and must now start managing with a stipend or rations; in some cases, they also need to physically be at the picket line and spend their time there picketing. Over-all, I would say that the cost of a traditional union during both conditions is proportionately spread out.

Selby Jr’s Last Exit To Brooklyn is a novel that captures the mentality of a particularly lethargic Union member (Harry) from several decades ago.

Now, take the case of the Alphabet Union: They don’t have any contract. But they do reserve the right to not work on projects that they don’t agree with:

All aspects of our work should be transparent, including the freedom to decline to work on projects that don’t align with our values. We need to know the impact of our work, whether it’s on Alphabet workers, our communities, or the world.

– Values, Alphabet Workers Union

At this point, the costs are no longer proportionately spread out anymore. The employer can’t do much in this case (certainly not an employer like Google) because these workers are not going on strike as a majority union. They are simply refusing to work on a particular project. Let’s call these projects that some people don’t want to work on, “morally disagreeable projects”.

For a large-scale company like Google, it will be extremely easy to move people around and find people to work on any project, their pay-scale and the prestige of working at that employer is high enough to balance out the “moral dilemma” caused by these projects. On the other hand, a smaller-scale company which wants to get involved in a morally disagreeable project will find it very hard to staff that project.

In general, an employer who wants to work on such a project has a few options to move forward:

Staff the project using people who don’t mind working on it OR
Stop working on the project

Option (1) will always be feasible for large-scale companies; they will certainly work for companies like Alphabet. Option (2) will be the most obvious option for smaller scale companies. If the company depends on this project for it’s survival, it might go under. This in-turn will have a ripple effect, that leads to a “bad rep” (a bad reputation) for places where such projects can not be undertaken. Leading to newer companies attempting to work on these morally disagreeable projects either not coming up at all or coming up in a different location and breaking up the Silicon Valley hegemony on start-ups.

Either way, it is quite clear to me that the existing smaller-scale companies and future start-ups, in the location where organizing similar to the Alphabet Union becomes popular will bear the brunt of the burden. This disproportionate burden on one party leads me to believe that there is a structural problem with the idea of a union where the costs are disproportionately shared.

Structural-concerns

As we have seen, the Alphabet Union is not going to negotiate a Union contract. Instead, they are going to exist as a body that is going to work on the sidelines. Google’s current rhetoric seems to be a complete lack of recognition for the union: Employers can insist that they will only deal with employees on an individual basis, as hinted at in this New York Times article:

In response, Kara Silverstein, Google’s director of people operations, said: “We’ve always worked hard to create a supportive and rewarding workplace for our work force. Of course, our employees have protected labor rights that we support. But as we’ve always done, we’ll continue engaging directly with all our employees.”

…

Although they will not be able to negotiate a contract, the Alphabet Workers Union can use other tactics to pressure Google into changing its policies, labor experts said. Minority unions often turn to public pressure campaigns and lobby legislative or regulatory bodies to influence employers.

– NYT article announcing the union

In the lack of a union contract, the benefits for membership are very little. One gains some power from the fact that this block of employees might work together to push the employer into some difficult arguments in the public sphere, but this would be no different from the public pressure campaigns that employees at several software companies have been running over the past year as facial recognition and software for border control agents have started becoming hot-button issues with this group of people.

Furthermore, one of the biggest problems that experienced organizers identify and talk about is the lack of incentive for the highly-skilled, focused-on-work employees to join traditional unions. These employees tend to have good relationships with their manager, good wages and a generally good opinion of their employer. They are the hardest to get to join the union, and they are the most important for the union, because in their absence, the union does not have legitimacy in the employer’s perspective. McAlevey talks about this at length in her interview on the Ezra Klein Show.

I believe that most software engineers work in firms where the difference in skill varies only by their grade, and rarely has huge variations within peers who are on the same grade. In this situation, either all the people from a given grade will have an incentive to join the union OR they will not join the union. My bet is that the membership numbers for the Alphabet Union will stall and they will not be able to sign-up even 10% of the total Alphabet employee base.

Finally, the other major structural concern that I came up with was the difference in the union’s use-case. Traditionally, unions have been used by workers who have been exploited by their employer. The ability to bargain collectively gives these workers the kind of leverage that they can’t get if they were to negotiate their work hours or pay with their direct manager in a 1-1. This technique has been effectively used by factory-floor workers, dockworkers, teachers to get better pay and better work hours. In all these cases, the underlying issue is related to a concrete Labor Law issue; this concrete connection makes the reasoning objective. On the other hand, a morally disagreeable project, is ill-defined and completely subjective. The lack of objectivity makes it very hard for the employer to predict which employees will not work on what projects (HR departments frown on discussions about political leanings at work, although there is wide variation on this between companies).

What is a “morally disagreeable project”?

(The subjectivity of this definition might be abundantly clear to you. If so, you can skip this section.)

The moral nature of a project and whether it agreeable to one or not depends on their political leanings and personal philosophy. This evaluation is bound to vary from one person to the next, and there is no real “control” group here that can be used as a litmus test to decide whether something is disagreeable to a majority or not (Elections are the closest we come to implementing this mechanism, and we often don’t get the result that we want, or even the result that the majority wants if the system has structural problems).

Going further, I argue that one would be able to find an argument against any company that harms either society or the environment. Several existing arguments can be re-framed as arguments that harm people or the environment.

In the case of Amazon, the impact of 2-day shipping on the environment has been widely studied. This Vox video condenses a bunch of the research and concepts into a short snippet.

It will get tougher for employers to adjudicate whether a given project will be morally disagreeable to the team that will work on it. The best case result of this is an employer whose limits match exactly with their employees. The worst case is one where the employer and the employees are on completely different sides of the spectrum, leading to either pandering to ensure that attrition stays low or doing what the employer wants to do at the risk of losing talent, due to a belief that what the employer wants to do is worthwhile.

Facebook is already facing challenges like this one and as Mark Zuckerberg puts it, “We should remember that Facebook’s employees are much further to the left compared to Facebook’s users”. As the employer, Zuckerberg and Facebook, the company need to focus on their users. But they are also caught in a bind with their own employees unwilling to work on a platform that they believe is empowering the people that they don’t want to empower. Casey Newton and Ezra Klein flesh this out further on the podcast, The Interface.

The border line between agreeable and disagreeable is shifting frighteningly fast. The discourse has moved a long way away from how we were talking about climate change or fossil fuel consumption reduction or even fossil fuel production compared to 2010, only 10 years ago. The change over the next 10 years might not be as dramatic, but there is no way to tell. Organizations that like to have 10 or 20 year plans and “roadmap” slide decks are going to have a nightmarish time figuring out how much things will change.

Over the coming months, we will see the saga of the unionized software engineers and other Alphabet employees play out. We will also see what happens when the moral alignments of an employer and their employees does not line up perfectly, by looking at the crisis that seems to be brewing inside Facebook. Both of these are new phenomena that are caused due to a simultaneous increase in the irreplaceable nature of some employees in these companies and the increase in the scale of these companies and the incredible variety of people that their products touch.

Strength of the Counter-Argument to Public Advocacy

2021-01-04T00:00:00+00:00

On Dec 10th 2020, Zeynep Tufekci made a public argument for a trial of the single dosage of the Pfizer and Moderna mRNA vaccines on her mailing list, Insight. When I first read the argument and it’s basis on the apparent safety and efficacy of just the single dose of the Pfizer/BioNTech vaccine, as described in the official peer-reviewed paper, I agreed with Tufekci. Her argument is a solid one and poses the central question: Is vaccinating 95% of 50% of available doses better than 60-80% of the total available doses? The answer is equally simple: No one knows yet. Some inferences can be drawn from the data of the 1300 people who didn’t get the second dose in the original N = 40,000 clinical trial. But this subset of 1300 people is not as random as the original well-designed trial. Another clinical trial, the advocacy for it in as many public forums as possible and subsequent funding / trial design seems like the prudent way to go forward, as Tufekci describes in her post.

On Dec 30th 2020, Tufekci posted philosophy professor,Matthew Noah Smith’s counter to her original argument on Insight. I read that post today and found it to be particularly weak. This post goes on to explain why I felt that, and the peculiarities in Smith’s post.

The post intrigued me because I was looking forward to a rousing philosophical debate on the greater implications of whether public advocacy was really flawed in high-stakes situation. Unfortunately, the post doesn’t go much beyond this particular case.

For context, the number of available doses hovers around 50 million for the US right now. This question is already being faced by several other countries (Indeed, the UK has decided to go ahead with the single dosage regimen).

As can be seen, the incidence of Covid-19 drops off dramatically about 10 days after the first dose (look at the chart on the top left). However, the trial is designed to measure the efficacy of two doses: a prime and a booster. Eyeballing (we don’t have the underlying data), the first shot seems to have 65-80% efficacy in the full group—but we don’t know for how long that lasts.

One might argue accurately that countries scarcely have the time to accommodate philosophical arguments, that the best thing to do right now is the thing that will reduce short-term and long-term adverse outcomes. I will table that for the time-being.

Putting that aside, here are some things that I couldn’t reconcile within Smith’s argument, that ultimately lead me to believe that Smith’s argument is weak.

The cautious optimism/overt pessimism fork in attitudes

Here are some quotes from Smith’s article:

It is more difficult to weaponize technical language used in academic journals and grant applications than it is to weaponize exasperated charges of epistemic irresponsibility made in places like Twitter. For this reason alone, we should restrict calls for single-dose trials to venues where experts engage other experts.

…

Bad actors with power have regularly harmed marginalized communities in the name of science. This same pattern of oppression could be reproduced when it comes to vaccine distribution. … The rhetoric around a single-dose trial could be twisted by people as a pretext for denying second doses [to] the marginalized. (sic)

…

This plays into the pernicious pandemic wartime imaginary we’ve all been invited to take up. Doctors and nurses are “heroes” who have been at the “front lines,” and hospitals are “war zones.” Once we couple this with the firm conviction among conservatives that the police are a beleaguered identity group, an easy strategic choice in response to vaccine distribution woes will be to deny poor communities and communities of color both doses of the vaccine until all those “battling” the virus receive two doses.

I find that the fork in Smith’s argument’s attitudes towards similar groups of people remains unresolved: Smith is very optimistic that the conservative media will not find an article or letter in the Journal of the American Medical Association (JAMA) or find it hard to exploit; while being extremely pessimistic about the outcomes of public advocacy: namely, a coordinated and racist effort to ensure that police officers and other “front-line” workers will be given a 2-dose vaccine regimen, at the cost of the poor, homeless, and other vulnerable demographics; all of whom also have the least access to healthcare.

This dichotomy is hard to reconcile. The conservative media and the government bureaucrats and lackeys that Smith is splitting up in his article are both on the same side of the political spectrum, and ostensibly very close to each other in terms of both social and economic policies. Why would one group be particularly restrained about legitimizing single-dose regimens using a JAMA letter, whereas the other group would use a public version of the same content, to actively prevent particular people from being vaccinated? One must argue either that they are all con-artists who want to find the slightest shred of scientific doubt, and use it to exclude people or that they are restrained, well-intention-ed actors, whose moral threshold for converting scientific doubt into a strategy, lies somewhere between a New York Times article and a JAMA letter. Smith seems to be arguing for the former, but this quote is an outlier and doesn’t make it too far.

All they need is quasi-scientific cover for unequal spatial distribution of a two-dose vaccine distribution.

Smith’s overt, almost militant, pessimism towards the prevention of the second dose for a particular group of people seems unnecessary and hard to justify: Even if it was just informal policy, any widespread attempt to deny second vaccine doses will quickly become a big story in the Liberal media. Moreover, states where the state legislatures are Democrat-controlled do exist (albeit, they are in the minority). Vaccine efficacy rates in these states will be much higher even when the same number of doses per-capita have been doled out; scientists and pundits will both be hard-pressed to find the reason and this (in)formal policy would be the primary suspect.

Are Technical articles harder to exploit for political gain?

While it is impossible to prevent cynical politicians from weaponizing science for short-term gain, simple precautions should be taken. One tactic is to avoid turning a scientific disagreement into acrimonious public disagreement. It is more difficult to weaponize technical language used in academic journals and grant applications than it is to weaponize exasperated charges of epistemic irresponsibility made in places like Twitter.

The second peculiarity in the argument is that Smith vehemently believes that it will be considerably harder to demonize the kernel of scientific doubt, inherent in everything, if that doubt is expressed in a JAMA letter, whereas it will be far easier to exploit if it was done through a post on Tufekci’s blog, a New York Times article, and a press conference at Harvard. Considering that the conservative media is made up of competent institutions which have researchers combing through public discourse to find chinks in the armor of the 2-dose vaccine regimen’s requirement, there is no doubt in my mind that they would find the JAMA letter. Given that base assumption, what makes a JAMA letter harder to exploit? The content would have been the same. Smith says the same thing, “All they need is quasi-scientific cover for unequal spatial distribution of a two-dose vaccine distribution.”, although this acceptance doesn’t affect his underlying argument.

Going one step further, I argue that the JAMA letter approach would be detrimental to Tufekci’s cause, as the journal article would have far fewer readers. It would not enter the organic news feeds that are consumed by people who don’t get their news from conservative media. They might even come across the article first through coverage of conservative media on other media outlets as a “Factcheck” or as “Media critic”, which substantially reduces the power of the advocacy.

The story that deserves to be publicized is not the one that the Conservative media took a JAMA letter and mis-categorized it. This would invariable be what goes into the feeds watched by Liberal news consumers. The coverage of the story will become the story, similar to CNN coverage of the Healthcare bill’s repeal.

In fact, in the quote at the beginning of this section, Smith seems to accept that it’s impossible to prevent cynicism, and that these would simply be measures that might work. Weighing the argument for a single dose trial (which could potentially double the number of vaccinated people) against the off-chance that cynical politicians and media outlets on the right will be sloppy and miss a JAMA letter, the former’s superiority, in terms of better, wider, positive outcomes, is clear to me.

Tufekci’s argument’s coverage will mislead non-medical observers

My final qualm with Smith’s argument is his belief that the stories covering the strategy that Tufekci took will be confusing or misleading, even if they don’t intend to be.

Even though the public message is that we should trial a single-dose vaccine, it is reasonable for someone unfamiliar with how medical research works to infer from a news story about the proposal that a single dose would be sufficient for vaccination. Furthermore, it is reasonable for someone who sees a story on TV about a study of single test efficacy to then describe it to acquaintances as a story about how only one dose of the vaccine is needed. After all, isn’t it rational for these non-experts to infer from respected professors aggressively advocating in public for a single-dose trial that a single dose would be sufficient? Why else would they be so vocal and so insistent on the urgency of a trial?

…

The intensity of emotions around these issues does not make trivial, often ersatz, scientific disagreement less likely to be perverted into a tool to discredit even the most widely accepted views.

…

What will happen if rhetoric supporting a single-dose trial is weaponized? It is difficult to determine. We have some evidence from both this pandemic and the anti-vax movement that non-trivial portions of the population might, on the basis of the weaponization of public advocacy for a single-dose trial, reject public health guidelines around vaccination.

This lies in direct contrast to something that he says just a few paragraphs before:

If a trial is warranted, then we ought to have some confidence that a single dose of either vaccine would be effective. But we also should have sufficiently low confidence that a single dose would be effective. This is exactly why a trial is necessary.

Smith seems to be saying that non-medical readers will somehow “miss the point”, even in good faith coverage of Tufekci’s strategy. I don’t think this is true or fair. The problem is not with the coverage, but with how most people consume the news: in short bites, as web page titles OR the first 2-3 minutes of a YouTube video describing the story. A consumer who reads Tufekci’s article, even just the first few paragraphs, will not come out believing that a single dose is better or efficient enough. To her credit, the article on Tufekci’s blog doesn’t even mention her argument in the title. It is a broader appeal for effective policy-making in the absence of sufficient data.

Most Valuable Resource

2020-12-13T00:00:00+00:00

A few days ago, I started thinking about why I was not spending time writing posts for this blog OR working on a side project that I have wanted to work on for a while. My conclusion was that I was not allocating my time properly. This is not original; I didn’t think it was a significant conclusion at first. As I started working on that, I came across another question that felt like a pretty good summary of what I was feeling at that point in time:

What is the Most Valuable Resource (MVR) in your life right now?

I started thinking about this off-and-on, and I eventually came to the conclusion that right now, it is Time. Thinking back to my childhood or my time in college, and I realized that my MVR in those stages of life were different from what it is now. In college, it was Experience. In school, it was Identity. This post is an exploration of my thinking on this subject. It is meant to stimulate readers to think about what their MVR is, how it has changed over the years, and how they might go about gaining more of their MVR through conscious changes in behavior.

High School: Identity
College: Experience
Now: Time
Caveats

High School: Identity

Writing about finding one’s identity in high school is not very interesting or useful. The discovering of one’s identity in the world, in relation to all the other kinds of archetypes, is a major part of one’s life. I think there’s some debate about when exactly one knows who they are and are able to accept and make peace with it.

When I was in my preteens, I had my first few experiences of being betrayed by the people around me that I had considered to be friends. I don’t remember the actual betrayals or how egregious they were anymore. My 13 year old self would vehemently claim that it was an injustice that one couldn’t simply understand, “It is what you live it”. Around this period, as I was trying to understand why people did these things, my parents would tell me often, and forcefully, that one must be ready to deal with people who are unlike anything that one was raised to be. This simple piece of off-hand advice has proven very useful, as I have read more about the thought process, conflict, political opinions and persuasion.

As I moved into high-school, I embraced the identity of a studious, mathematics nerd. I installed Linux Mint on the desktop computer that was in our home, and waited with bated breath for the progress bar on the screen to reach 100%. I was waiting to find out if I had effectively prevented my parents from using the computer through the weekend, while I figured out what had gone wrong, and tried to fix it by reading help web pages on my Dad’s work computer. Or, if I had successfully installed an OS alongside Windows. My initial installations were dual boot installations where Mint was installed inside Windows as a simple program. As I gained more confidence, I started installing, uninstalling and re-installing various Linux distributions in an effort to somehow get to a “not particularly nerve-wracking” installation experience. I did not succeed. (These days, I get around this by never removing any installation of Linux, from any computer that I am using; a rather unappealing cop-out)

This initial exposure to Linux built a familiarity with the command line from a fairly young age. I started using Git with absolutely no understanding of what it was really for back in 2011. None of these things were particularly big leaps in my search for an Identity, but they all added up to something very concrete: I was not uncomfortable having interests that others didn’t share, I was not really worried about conforming, I was not worried about appearing uninterested in the “elementary” computer science class, as I corresponded with my CS teacher, via e-mail, trying to understand how I could use languages other than C, and what she thought would be useful “Next Steps”.

(_Editorial note:_ Reading this now, I am worried that I thought of myself as too much of a non-conformist, when I was absolutely not anything like that. I had good grades throughout all my school years, and complaints from teachers to parents about me in school were never about not conforming or question my teachers.)

This self-identification as a math nerd stuck to me as I moved into the new environment of a coaching class with younger, more approachable “professors” and a group of extremely motivated peers who were focused exclusively on understanding the material and performing at the competitive, university entrance exams.

Going into college, it continued to held sway. In college, I gravitated towards courses in the Mathematics Department, even though my major was in Mechanical Engineering, a discipline that I never came around to fully like. Eventually, I ended 5 years of college with a minor in Mathematics and Computing, consisting of courses on the Foundations of Cryptography, Modern Algebra and Graph Theory.

(_Editorial note:_ I had to look at my transcript to remember the name of the Algebra course)

During this period, I was mostly reading fiction and I don’t remember anything that I came across that stuck with me for a long time. Looking back, these 2 quotes about studying for examinations and performing at school and other academic settings seem apt.

Don’t try to be at the top of the class. Don’t be at the bottom either. Find the 85% scoring sweet spot where you know the material well enough that you are spending the least fraction of your time on mandatory schoolwork. Spend all the remaining time on things that you want to do.

People care about your grades only when they are bad.

(The first quote is from a very famous essay about preparing for exams. I have read this essay a couple times now, but I can’t find it right now. If I find it later, I will edit this post and put a link here)

College: Experience

I studied in Kharagpur, West Bengal in India. Kharagpur is a mid-sized town and is located a 2-hour drive or a 3-hour train ride away from Kolkata, one of the four major metropolitan cities in India. Until this point, I had lived in an industrial township near Mumbai and then in Navi Mumbai proper, so my experience until then was very much of a bustling metro city with a wide variety of things to do (like going to a movie theater with my friends after school exams).

Kharagpur is a great college town. The college’s campus is a physical gated town within a town. One ventures out more and more often as one gets used to living in the town. In my final 2 years on the campus, I left the campus to go to the spots around Kharagpur town (restaurants and such) more than the 3 years before that combined.

While this sounds like a good set-up (and it was), I craved Experience. In particular, I craved the experience of foreign travel. I was hooked to Casey Neistat’s videos about his life as a traveling Ad film-maker. I left India for the first time a few days after my twenty second birthday, on a trip to Bali with a friend. This trip was memorable and surprisingly unremarkable. I had incorrectly believed that my first trip abroad will be set in stone, and end up defining all my trips after that. This was not the case, and my mind moved along much faster than I had originally imagined it would.

There are two quotes from this period of time in life that really resonated with me right after I had achieved my goal.

The greatest things that happen to a human being will happen to you too; you just have to lower your expectations

– Phil Dunphy (Modern Family) (A fictional character)

You will get everything that you want in life; but it will take just a little bit longer than you expect it to.

– Casey Neistat

Now: Time

Now, I have a full-time job as a Software Engineer. I live in a foreign country, I am studying a foreign language. The resource that has become my MVR is Time.

Don’t say “You don’t have time to do X”. You did have time. You decided to do something other than X in that time.

I noticed this feeling of reduction of time early this year when I started working from home. My commute was an hour long, so one of the first immediate consequences I expected was an increase of 2 hours per day in my daily routine. This didn’t happen.

After 2 months, I realized that this was not happening and I looked closely at my work hours and found that they had not increased significantly (They had increased slightly compared to before, but certainly not by 2 hours a day). My sleep patterns were unchanged according to my Fitbit data. I was sleeping the same amount of time, and I was mostly adhering to a routine that allowed me to get off of work around 7 pm.

I started focusing in on this block of time after 7 pm. I was going to sleep every day around midnight, or later. That was a solid 5+ hours that I was using to do something other than read books, write emails and essays, and cooking or cleaning. The culprit was Netflix and other on-demand video sites. I found this anecdotally, and quantified it by looking at the amount of texts that I was sending my friends about new Netflix series that I had started watching. This was a non-issue when I had a commute as I was tired by the time I got back home and I rarely watched new shows.

For a period of time, I didn’t think this was particularly concerning or noteworthy. I was watching Netflix voluntarily and I knew that I could break the habit off if I needed to. I had also come across some really good series like Ozark and Parks and Recreation.

As I started preparing for a proficiency test for Japanese, I started reading Japanese novels. These books were extremely difficult to read and they often took over 16-20 hours each weekend. For my first few novels, I spent every waking minute during those weekends that I was not cooking, cleaning or eating, reading the novel. This was a very rewarding experience.

From here, I started thinking about cutting down on on-demand video through out-of-band solutions (out-of-band = things other than canceling my Netflix subscription). A solution that I have been running as a trial for the past 14 days is a “no-browser-after-10pm” policy. Today was the 14th day of the trial, and I have decided to extend the trial for another 14 days as I have not noticed any adverse effects from not being able to use a browser (Firefox) for the last few hours of each day.

Caveats

A keen reader may have noticed that I didn’t mention Money as an MVR at any stage in life. This has been a great stroke of luck, and I have been incredibly fortunate to have not had to consider Money my MVR at any point in time.

This concept is something I came up with off-hand, along the lines of the “Most Valuable Product” (MVP) culture, that is common in some software engineering teams and companies.

Configuring Emacs - 7 months later - Act 3

2020-12-01T00:00:00+00:00

Note: This is the final post in a series of 3 posts about my experience configuring Emacs over 7 months, starting from scratch.

Emacs Lisp is the language that is used to configure Emacs. It is a functional programming language and looks very similar to the prefix notation that is used when doing simple arithmetic:

(/ 2000 (+ 34 4.5))
51.94805194805195

;; Insert the current time string at point in the active buffer
(insert-string (current-time-string))

Act 3: Writing Emacs Lisp Functions
- Delete all comments from a buffer
- Copy the contents of the current buffer

Delete all comments from a buffer

e501bb0a

My goal at this point was to write an Emacs function that would delete all the comments from a file. This goal was a fairly simple one (I thought) because I was using a Regular expression in Vim to do this: :g/^#/norm dd. I used the # because I deleted all comments only from shell files which were copies of the commands that I had run (My PS1 starts with a hash to ensure that when I ran this command, personal information like my host name or the path will be stripped).

In late October, 6 months and 9 days after I made my initial commit, I was able to write a function which actually deleted all comments from a buffer.

I faced some interesting challenges when I was writing this function. My original goal seemed extremely simple: Delete a line if it is a comment; run this for the whole buffer. Thinking like an editor, I decided to do this on a line-by-line basis.

One of the interesting challenges was figuring out when I was at the end of a buffer. Emacs does not seem to have any function to get the last line. There are functions to get the last byte position, but none to get the last line, that I was able to find. So, I had to manually keep track of whether I was deleting a line. This lead to a function which would call another function and depending on whether the line was deleted, subtract one from the original last-line-number value.

This process was full of other challenges that were related to my inability to find the name of the function that I wanted to use quickly. Emacs functions are not all named using some common pattern, which would make them easier to find. For e.g.:

line-number-at-pos returns the line number at the current position
(beginning-of-buffer) moves the cursor to the beginning of the buffer, (forward-line) moves one line ahead
(kill-whole-line) deletes the current line.

As you can probably see, there’s no common convention here. All functions that deal with lines don’t start with line-, they simply have the line word in them!

Copy the contents of the current buffer

The goal for my second function was to copy the contents of the current buffer to the system clipboard. Writing this function took far less time, and I was able to find the functions I needed to use fairly easily. It was also much smaller; in fact, it had only 5 lines:

(defun copy-buffer ()
  "Copy the complete buffer to the system clipboard"
  (interactive)
  (kill-new (filter-buffer-substring (point-min) (point-max)))
  nil)

My main problem here was the terminology. The Emacs terminology for everything is different:

yank – paste
kill – cut, that is, delete and copy to the kill ring

What I wanted to do was copy, I could find anything close to this in the Emacs glossary. After reading some more, I realized that the kill-ring was similar to the clipboard and pushing things onto that ring is the same as copying something. And that is how I reached this function kill-new. This function pushes it’s first argument to the ring => essentially copying that string.

Configuring Emacs - 7 months later - Act 2

2020-11-21T00:00:00+00:00

Note: This is the second in a series of 3 posts about my experience configuring Emacs over 7 months, starting from scratch.

After figuring out how to install packages, as you start to use the editor more, you run into cases where the keybindings are inconvenient or don’t exist at all. During this phase, the 3 months between June and August, I added a lot of keybindings. Most of them were to mimic the functionality that I had configured with Vim over some time. A lot of them were new keybindings which made sense only in the Emacs world.

At this point, I was using Emacs everyday for non-code related editing. Towards the end of this period, I installed some major modes which allowed me to program in Go within Emacs, but my first attempt at the installation of these modes was terrible and I had a very bad experience. After giving up for a short period of time, I returned to the problem once again and figured it out the second time around. This story is longer and I want to keep it for another blog post.

Act 2: Keybindings and Package Integration

Navigation using `Ctrl + H/J/K/L`

c458cc48

(general-nmap
  "DEL" 'evil-ex-nohighlight
  "C-h" 'evil-window-left
  "C-j" 'evil-window-down
  "C-k" 'evil-window-up
  "C-l" 'evil-window-right
  )

Navigation between various splits using Ctrl and the Home row keys was another one of the base requirements for me when I started using this editor. After installing general.el and reading evil-guide 10-20 times, I was able to figure out how to add the appropriate bindings for moving around splits using general.

The most confusing part of this were the quotation marks. Emacs allows functions to be specified with #' or with a simple single quote. There does not seem to be any difference and you can use either of those forms.

Keybindings to comment code

99683e0f

;; 24. Keybindings that use the leader key functionality in normal and visual mode
(general-create-definer leader-def-visual-mode
  :states '(visual)
  :prefix ","
  )
(leader-def-visual-mode "c SPC" 'comment-or-uncomment-region)

(general-create-definer leader-def-normal-mode
  :states '(normal)
  :prefix ","
  )
(leader-def-normal-mode "c SPC" 'comment-line)

This was the first section of the configuration which was more than just using a built-in function directly. I was starting to understand what these functions were doing and I was able to get the leader key behavior that I have always liked for commenting and un-commenting code in Vim. The comment-or-uncomment-region function is a Emacs-only feature which inherently understands the comment pattern for the language that is being edited.

Hooks

eae9cc91

;; 26. Remove trailing whitespace characters from all files
(add-hook 'before-save-hook 'delete-trailing-whitespace)

Hooks are a major part of Emacs. Until now, I have found that they are the main pathway for connecting modes with each other (e.g.: Run the spell-checking mode when the file type is Org). They are also very useful for doing things like the above configuration block, which deletes trailing white spaces from all file types before saving the file to disk.

Mode maps for Keybindings

71c9a782

-  ;; TODO: This should be active only in the org mode key map
   (ctrl-keybindings
+	:keymaps '(org-mode-map)
	"C-c l c" 'org-cycle-list-bullet
+   "C-c e" 'org-table-edit-formulas
	)

+  ;; 23. SQL format highlighted region
+  (general-evil-define-key 'visual sql-mode-map
+	"gq" 'run-sqlbeautify
+	)

This is another Emacs-only concept which was hard to understand for me when I tried to map it back to the knowledge I had about editor configuration already. After a few months of getting used to this, I am much more confident about adding new keybindings. For e.g., when I started testing elfeed today, I added several keybindings within the first few hours of using it:

1c672f7a

;; Elfeed Search's default mappings work are for Insert mode only
(general-evil-define-key '(normal visual) elfeed-search-mode-map
  "RET" 'elfeed-search-show-entry
  "u" 'elfeed-search-tag-all-unread
  "r" 'elfeed-search-untag-all-unread)

This evolution from not understanding what a mode map means or does, to understanding and being able to use it properly was satisfying for me. This process takes time, and if you are switching to something new and are having trouble not understanding things that everyone else treats as basic, I would tell you to simply stick with it for a few more months and see if you understand it better then. (Johnny Harris has a great video about how he figured out how to photograph far-away galaxies that runs along the same lines.)

Configuring Emacs - 7 months later - Act 1

2020-11-15T00:00:00+00:00

Note: This is the first post in a series of 3 posts about my experience configuring Emacs over 7 months, starting from scratch.

My first commit to icyflame/.emacs.d, the repository with my Emacs configuration, was in April this year (7 months ago). Now, I have reached a point at which I have cleaned up my configuration enough to feel okay about making it open source. In this post and the two posts after this one, I will go over some of the most important steps in my configuration’s evolution from scratch to it’s current form.

Context

I started my configuration from scratch with a vanilla Emacs installation. This is a personal preference; there are several configurations out there which allow people to start from a opinionated configuration which they can understand and tweak to their needs. (There are even Emacs distributions like Prelude which have a lot of the plugins which I configured manually already installed!)

This post is not really meant as a guide for the reader to get started with Emacs or copy my configuration and use it as is; that is not my goal anyway. My main goal here is to talk about the things in my configuration which feel obvious to me now, but were particularly difficult to figure out when I initially tried to figure it out. Along the way, I will also try to share some of the fun things I discovered about Emacs Lisp, a language that is equal parts fascinating and obscure.

That said, if you plan to use parts of my configuration and are unable to figure out why something is not working, I would love to hear your questions and help with figuring out the reason!

Act 1: Fundamentals

Act 1: Fundamentals

Right after installing Emacs, I wanted to figure out a stable method to install new packages from using Emacs’ package management system. I wanted to ensure that this would be done using configuration, and not using ad-hoc commands. My motive for this requirement was mainly the ease of migrating this configuration between my personal and my work computer (I started using Emacs on my personal Linux machine at home; I use a Macbook Pro at work (from home)). For the 2 months between February and April, I was using Emacs everyday without any major changes to the base empty configuration that every installation comes with. This was not a very good experience, and I kept switching back to Vim whenever I had to do anything that was slightly more complicated than typing text.

Package Management

I got the require-package function from purcell/emacs.d. Back then, I didn’t fully understand what this function was doing. But the promise of this function was that it would provide a portable way to install packages from Emacs’ package registry, Melpa.

;;; from purcell/emacs.d
(defun require-package (package &optional min-version no-refresh)
  "Install given PACKAGE, optionally requiring MIN-VERSION.
If NO-REFRESH is non-nil, the available package lists will not be
re-downloaded in order to locate PACKAGE."
  (if (package-installed-p package min-version)
	  t
	(if (or (assoc package package-archive-contents) no-refresh)
		(package-install package)
	  (progn
		(package-refresh-contents)
		(require-package package min-version t)))))

Evil mode and general.el

4d083a63

;; 12. Install general package
(require-package 'general)

;; 13. Add key mappings for common actions using general
(general-evil-setup)
(general-nmap
  "DEL" 'evil-ex-nohighlight)

After figuring how to install packages, the first package I installed was evil. This package brings in most of Vim’s keybindings. Installing this package right at the beginning is a double-edged knife, as it simultaneously makes Emacs more familiar and contributes to making configuring Emacs harder to figure out. Evil’s configuration options are, for the most part, exactly the same as Vim. But the parts that are different are hard to understand and I was flummoxed by these differences for about 2 months. During this period, noctuid/evil-guide was my main reference. The Readme of this repository is comprehensive and explains everything about Evil mode. I read this Readme several times to figure out exactly what was different, and how I could get rid of these differences completely.

general.el was a harder choice because I was not quite sure what this package did, but it was highly rated and everyone seemed to be using it. evil-guide talked about this package with high regard, and claimed that it would make configuring Evil mode easier. So, I took a leap of faith and decided to install it.

Don’t install Helm

Installing Helm early on was a mistake. Helm is an extremely powerful package, that is very hard to use and understand. It is often referred to as a replacement for Ctrl-P, but in reality, it has a lot more bells and whistles. These extra features make it very hard to understand where Helm ends and where Emacs begins. Using Helm within an editor whose keybindings you are not yet familiar with is hard. Personally, I think this was a big mistake and delayed my configuration efforts by about the 2 weeks that it took me to realize what was going on and uninstall Helm.

Org mode

22187304

;; 18. Org mode settings
;;; Set the done time for a TODO item when moving it to DONE
(setq org-log-done 'time)

(setq org-todo-keywords
	  '((sequence "TODO(t)" "WAITING(w@/!)" "|" "DONE(d!)" "CANCELED(c@)")))

I started using Org mode around April 28 when I made a commit to introduce the TODO states that I wanted to have in my Org file. This was right after watching the original talk about Org-mode. I am glad that I started using this early on as I was able to understand a lot more about how Emacs works as an editor (For e.g.: keybinding maps per mode, the global mode, minor modes, mode hooks) by configuring Org-mode, a non-programming mode whose configuration is much easier to understand.

Discovering Org Mode

2020-10-25T00:00:00+00:00

I talked about some of the problems with Markdown that I was facing in my daily note-taking at work and at home. I had been looking for a replacement for my note taking system since early this year. Org mode seemed like a good candidate and a few people recommended it to me, so I started using it.

Now, I have used Org mode for about 6 months at work and at home. In my personal life, I managed the move to a new apartment and switching to a new mobile phone provider entirely using a single sub-tree in my notes file. At work, I have used Org for a few months now as I worked on a variety of Search and SRE tasks; a lot of them included huge files and ad-hoc Perl scripts which were written almost entirely within Org and are still stored only inside Org. This post is about my experience with Org mode and a list of the things that I believe are the strengths and weaknesses of this markup language.

What is Org mode?

Org mode is a major mode for Emacs. A major mode is like a filetype in Vim. It has a bunch of special formatting logic and functions that only work in that mode. Emacs is always in a major mode.

Org mode also refers to the mark-up language that can be written within this major mode. This mark-up language is a lot like Markdown if you look at the plaintext. For example, here’s a quick sample with some common things that markup languages can do:

* Title
** Subtitle

Normal, *bold* and _emphasised_ text.

- Lists
- look
- the
- same
  - even with indentation
  - and list items with embedded code are dealt with properly
	#+BEGIN_SRC php
	public function getID() {
		return $this->id;
	}
	#+END_SRC

| Tables | Look |
|--------+------|
| The    | Same |

As the markup is similar to Markdown, it’s extremely easy to switch to Org from Markdown. In fact, you don’t even need to think a lot about whether you are writing Org or Markdown except for headings where the character changes from # to *. So, there’s nothing particularly special about the markup language.

Opening this file in Emacs won’t be very impressive either:

The file looks slightly better; this is only stylized Markdown though.

Org mode really starts differentiating itself is when you start editing it. Here’s my quick, 3-part pitch to get people intrigued and try it at least once:

Lists: Org mode (the layer of code sitting between the plaintext and your editor) inherently understands your list. So, when you type Alt + Return, it will create a new list item.
- If you are in a numbered list, it will check the numbers of all your list items from the top and fix any
discrepancies
- Alt + Up arrow will move the current list item up; once again, if you are in a
numbered list, the numbers will automatically be adjusted for all list items at that level
- If the list item you are moving has child items, they will also be moved alongwith the parent item!
- Editorial note: I was writing this blog post in Markdown, but the nested list item that I was trying to create using Alt + Right arrow, didn’t have any effect. So, I copied the text over to an Org file and pasted it as is. After doing that, I just had to do Alt + Enter to align the list items and start editing the text as Org mode text.
TODO list management, heading filtering: A common notes file has a lot of heirarchy and notes at several levels. Some of these notes will be TODO items that need to be done, or were done in the past. Org mode has in-built management of these lists using plaintext markers.
- Like other TODO management software, there are a lot of ways to visualize the things you need to do
- There are even some features that allow you to schedule TODOs in the future and then look at your day’s agenda (I have started using this feature recently)
Tables: Tables automatically align when you press Tab. I have not seen any application except Emacs which can do this so well. There are several GUI applications which can do this well, and theoretically, you can run them in Vim mode and get the same result.

Strengths

I thought about what I wanted to put here for a while. I finally decided to put the features of Org that I use daily. Org can do a lot of other crazy things but I have found that I don’t use them as much. (For eg: Org tables have a spreadsheet like feature which allows you to write formulas, but LibreOffice’s Calc blows this out of the water. So, I did not use that feature as much within org)

Lists, Tables, TODOs

Like I said above in my quick pitch, Org mode understands Lists and Tables. By understand, I mean that it understands Lists as computers understand arrays. For eg, it knows that the top element can’t be moved above it’s current position because it’s already at index 0! For sublists, it has (probably) something like a pointer inside it which lets it know that when the parent pointer is moved up, the sublist should also move.

This understanding of lists allows it to do things like change the leading character of a list to anything that you want with a simple keybinding (Shift + Left/Right arrow). A program like Vim is working with Lists using regular expressions. These expressions are bound to run into corner cases which will be fixed with a more granular Regex … ad infinitum. My most common use-cases for lists:

List of PRs for a given feature (the order in which they need to be merged, etc)
Release plans (say: to be shared for review)

Note: Org mode is pretty good list numbering tool. If you have a list of lines with content that you would like to number for some reason, you can do this by prepending the whole list with 1. and then using Org’s “Fix list” bindings (Shift + Right/Left) to number the list properly!

With tables, this behavior is pushed even further as Org behaves with tables almost like any Spreadsheet software. It is better than them, in my opinion, because of it’s ability to quickly change the width of all columns based on the content. With spreadsheet software, one would have to use the mouse and extend the column manually. All of this done with the TAB key: Type | CONTENT | CONTENT | <TAB> and a table is started and each column is the width of the column with the longest content.

With Vim keybindings, adding a column is done by simply highlighting the table and using block editing (Ctrl-V) to append or prepend a |. My most common use-cases for tables:

Pros/Cons list when comparing options / designs / implementations
Use-cases for a given feature
Comparing various external libraries/services

I have also often converted between lists and tables:

From: <CONTENT>: <VALUE> To: | <CONTENT> | <VALUE> |

This is fairly easy if the list is not nested. As tables look better, there’s a constant temptation to express information properly as tables, rather than lists.

Embedding code

#+BEGIN_SRC php
$ docker run -it --name=php php:latest php -a
Interactive shell

php > $password = "t";
php > $h = password_hash($password, PASSWORD_DEFAULT);
php > echo $h;
$2y$10$F6exUnYVDd.W4DSs0I8fdOssGPdj57L3pvmjef5oHEg8qdoahTONS
php > var_dump(password_verify($password, $h));
bool(true)
php >
#+END_SRC

The markup for embedding code is verbose in Org. It consists of #+BEGIN_SRC and #+END_SRC along with the language of the code that is embedded. I am still trying to figure out if there is some way to embed using Markdown’s `. Looking past the markup, this code renders with the proper syntax highlighting for the embedded language. So, within an Org file, you can have code from any language and it will all be highlighted according to that language’s highlighting rules.

When exporting to PDF or Markdown, the embedded code is also exported as expected. I have had some issues with several embedded code blocks that follow one another though.

Exporting to Plaintext / Markdown / ODT / Docx

I work in a company with a few different platforms for sharing information. We use Slack, Confluence, Jira and Google Docs. There is no underlying reason for the tool that will be used in any given meeting or project and it often varies depending on the people that I work with.

This gives rise to the need for a documentation tool that will export to all of those formats. Here, Org mode blows Markdown out of the water. Exporting out to another format is a first-class feature in Org mode. There are some really good exporters that are included by default. Of these, I have found Plaintext and Docx to be the most portable!

Note: Docx files can be created using Org’s ODT export function. ODT is used as an intermediate format. The .docx file can be imported into Drive and opened using Google Docs. Google Docs now has a dedicated “Docx”-supported mode which allows you to collaborate and edit a Docx file!

Org’s lists and Tables can be copied as is and pasted into Slack or Confluence in their code blocks section. They look good on any screen that’s not a phone screen! This allows you to export without any other tool, by simply copying the markup.

For more involved exporting (like text with Embedded code), I use the Plaintext export to get a UTF-8 buffer which I can copy and paste into Confluence / Jira / Slack’s text snippet or embedded code markup object! The plaintext that comes out of the Embedded code is particularly good looking. Embedded code is wrapped on the left with a box made of - and | characters!

Testimonial: When I shared my notes from an important internal HR meeting widely as a plaintext snippet on Slack, one coworker sent this message to the channel:

Your notes are beautiful.

I have shared notes written in Markdown several times before this, but I have never gotten this kind of a message! Clearly, the plaintext export is generating very good looking files. (This is the file generated when I exported to UTF-8 plaintext from this example Org file.)

Weaknesses

Single notes file

I have been using Org mode for about 6 months now and perhaps the most annoying weakness for me is that I have put all my notes in a single file. Now, due to Org’s intuitive hierarchical structure, it will be extremely easy to move the notes to separate files by project. But I have not yet figured out how to get Org mode to search across several Org files. For now, as all of my notes are in a single file, I have to be extremely careful about destructive actions that I take within this file.

I have a script which commits to Git every 15 minutes. I started doing this after I lost about 20% of my notes file during a bad edit in which I deleted a ... block. When I deleted it, I had no clue that this ellipsis represented a block of Org and was not just something that Org added to my underlying text file. I am still trying to control my instinct to delete lines that have only an ellipsis.

Strange export behavior

There are some “properties” which can be set on every title / heading. These including some useful ones like EXPORT_TITLE and EXPORT_AUTHOR. Unfortunately, I haven’t really gotten them to work properly until now. Often, when I am trying to export a single title, it will not use the EXPORT_TITLE that is provided in the heading and instead use the title from the file’s properties!

These Export properties are probably being used by the actual driver which handles the export from Org mode to the appropriate format. So, some drivers do handle this properly, whereas some don’t! (For e.g.: EXPORT_OPTIONS is a common property which can be used to turn off subscript/superscript support (useful when writing notes with variable names) and turn off the Table of contents (useful when exporting to Plaintext); this property doesn’t work with the Plaintext exporter)

Building and running GnuCash in Docker

2020-09-13T00:00:00+00:00

Introduction

I have written about GnuCash a few times before. I have been using GnuCash version 2.6 for nearly 2 years now. A lot of the features that were added after this version weren’t really useful for my use-case, so I never upgraded. Also, the binary that is distributed through the Ubuntu App store / apt-get repositories is v2.6. Versions after that were not published to the aptitude repositories, for some reason. (A lot of software packages do this)

Recently, I ran into a very strange bug. I wanted to report this bug to the GnuCash team and get their feedback on if this was something that they had seen before. To avoid reporting a bug on an old version which was no longer being supported actively, I got the latest version of the GnuCash code and started a long, uphill battle to build it locally! This post is a summary of that experience.

For those who are looking to run the latest version of GnuCash inside Docker, you can get this Dockerfile and build the Docker image. The command to run it on an Ubuntu 18.04 LTS host is given at the top of the Dockerfile.

Tools

GnuCash has some documentation about building on Linux. It starts with this page called Building On Linux. Going from this page, I kept jumping to several subsequent pages which felt strange and I was soon confused about what I was looking for in the first place.

This is the short summary of what I learnt:

GnuCash is written in C
Building the software has two distinct steps:
Configuring the build: For this, we use Cmake
Build (for real): For this, we use Make
The build directory must be completely separate from the source directory
```
 $ tree -n -A -L 1
 .
 ├── gnucash-4.1
 ├── gnucash-build
```
This is the structure that I used. We will be running all of the commands inside the build directory. In this example, gnucash-build/

Required packages

It took me quite some time to figure out exactly where all the files should go and what came first, etc. Once I had figured out the purposes of Cmake and Make, things got a little bit easier.

I started with this command:

(gnucash-build) $ cmake ../gnucash-4.1

This command will most probably fail, unless you have all the dependencies installed. If this command does not fail, then you can switch to the next step: Build.

For me, this command failed several times. Each time it failed, it output either an error with the name of a package that was missing or a warning with a particular part of GnuCash that it couldn’t configure properly because of missing dependencies.

Warnings

I had to remove two parts of GnuCash to get past the warnings: AQBanking and SQL support. After seeing the warnings, I wasn’t quite sure how to disable them. A quick look at the CMakeLists.txt file helped me figure that out. I don’t know how to read CMake files, so I just winged this part. I found a few mentions to the actual option and found these lines:

option (WITH_SQL "Build this project with SQL (libdbi) support" ON)
option (WITH_AQBANKING "Build this project with aqbanking (online banking) support" ON)

After finding this option, I discovered that cmake takes configuration options using -D

-D <var>[:<type>]=<value>    = Create a cmake cache entry.

-D’s description is confusing, but it’s the option we want to use.

With this, I reached the command:

(gnucash-build) $ cmake -DWITH_AQBANKING=false -DWITH_SQL=false ../gnucash-4.1

Running this on my Ubuntu 18.04 installation gave me an error mentioning the name of a missing library. I went ahead and installed the library and ran cmake again. I repeated this step several times.

Errors

Cmake configures the build by ensuring that each library that is required is available on the host and is in a location that can be found by Make (or it will tell Make to find it wherever Cmake found the library). As I saw each warning, the name of the library I needed to install was either clear to me from the error or I had to run a search on packages.ubuntu.com.

I had 22 required packages missing. The complete list of packages can be found in the final Dockerfile’s apt-get install command here. This step took the most time at about 90 minutes. After 4-5 steps, the process was pretty clear to me. A couple of things that I learnt during this process:

For building software, almost always the -dev versions of packages are required. For example, I had the package libxslt1.1 installed. This wasn’t enough though. Cmake kept telling me that the xsltproc could not be found. This was because I didn’t have libxslt1-dev package. After installing the -dev version of this package, I was able to go to the next missing package.
You can check what packages are already installed using dpkg -l. Using a tool like grep allows you to quickly check whether a package is missing or if there’s some sort of version conflict: dpkg -l | grep xslt
For most packages, I was able to install them fairly easily. For 2 packages (libglib2.0-dev and libxslt), I was not able to install the required package for a few minutes. After moving around searches, installing and removing packages, checking dpkg -l output, and reading the error message several times, I was able to install the required package.

Build

The actual build part of this process is extremely easy. Cmake ensures that everything that was required has been installed and can be found in their usual locations. If not, Cmake will tell make where to find them using the Makefiles that cmake generates when the configuration process is complete.

(gnucash-build) $ make

This command will run for a considerable amount of time. On my laptop with an i7 8th gen processor and 16 GB of RAM, make ran for about 20-30 minutes (I didn’t track this closely; I was relieved to have reached this stage and I didn’t really care even if this command took a few hours as I was probably not going to run this command again EVER)

When make is done, your gnucash binary should be inside gnucash-build/bin. There will be some more binary files.

(gnucash-build) $ ls bin
gnc-fq-check  gnc-fq-helper  gnucash      gnucash-valgrind
gnc-fq-dump   gnc-fq-update  gnucash-cli  jenny

(gnucash-build) $ ./bin/gnucash --version
GnuCash 4.1
Build ID: 4.1+(2020-07-25)

Conclusion

I have built most of the small utilities that I use on a daily basis (like my recent favorite, git-fzf). GnuCash was one of the major parts of my computer usage and I am glad that I can actually build the software which is able to parse the huge .xml.gz file which contains all my accounting information!

On Hope and Independence

2020-08-01T00:00:00+00:00

Note: This post is different from the typical posts that I are on this blog. It is about something that made me think when I watched a TV show recently. I will explain the set-up of a story I heard on the show. After that, please take some time to think about what you would do if you were the character in that story. Then, move on to the next section which will show the original ending of the story, and the ending that came to my mind right before I heard the original ending on the show (there’s a pause in the show).

The set-up

Once upon a time, there was a girl who was very sick. There was a tree outside her room that she saw everyday through a window. Seeing the leaves on the tree fall off, she told her father that when all the trees had fallen off of the tree, she would die too.

Her father, hearing this, decided to do something about it.

At this point, take a moment. What would you do if you were the little girl’s father?

…

The ending

He went to the tree when there was only one leaf left on it. He stuck the leaf to the tree with glue, ensuring that it will never fall off.

Every morning, the little girl would wake up and see the leaf on the tree. Seeing that one leaf hang on for life, would rejuvenate the little girl’s will to live.

An alternate ending

When I heard the set-up of this story, I thought of a very different ending. I did not think about what the father in the story did at all. This is what I thought the father would do:

Seeing his daughter connect her will to live to the tree outside her window, he realized what he needed to do. The next morning, he took his daughter out to the tree and said to her: “This tree, it has nothing to do with how long you will live. You will live a long, happy, full life.” He started plucking the leaves one by one from the tree. His daughter panicked, she looked at him, not understanding why her father was doing this. Her eyes filled with tears.

In a few minutes, her father had plucked all the leaves on the tree except for one. He took his daughter’s hand and brought her closer to the tree. He asked her to pluck the final leaf. The little girl, afraid of what might come about, refused to do so. Her father took her hand and wrapped it around the leaf and pulled it from the tree. The leaf was in the little girl’s hand.

Looking at it, the little girl understood that she did not need the tree to continue living; everything she needed was within herself.

Conclusion

There are a lot of stories that are meant to give people who have lost hope a chance to regain it. Constantly, we hear them in TV shows, on YouTube videos that are released in the last week of every December, from journalists who have seen something heart-warming in the toughest of terrains. They are all trying to desperately convince the viewer that “The searches for hope outnumbered the searches for fear.” (something from one of Google’s “Year in review” videos)

I think they are meant to enforce the narrative of being a collective, of being together, of fighting crises by joining forces, of not being alone. Allowing anyone to connect their life to something outside of themselves seems unnecessary. Why should the little girl have to depend on the existence of leaves on a tree to continue living?

Source

Delhi Crime, a 2019 Netflix show; Season 1, Episode 6, 36:30

How to Take Smart Notes (Ahrens)

2020-07-09T00:00:00+00:00

I read this book after seeing Sreejith’s 5-star review. It was a habit-changing book for me.

The most important take-away for me was the difference between a writer and an archivist. As I have been reading more and more non-fiction books in the past couple years, I realized that there were several connections that I was noticing but wasn’t really making on paper anywhere. I was making notes about separate pieces of literature, but I wasn’t really connecting them. My behavior was that of an archivist.

To move into a writer’s mindset I realized that I have to start thinking about how anything I read is changing my mindset and actually write my thoughts down (apart from the things that have already been said by the author).

A few other things that were expressed very well in the book:

Writing is everything; everything must lead to writing; the final written piece could be a research paper or something that you never publish, etc; but everything you do around “research” must have a single goal: To write something about it
Writing is the process of understanding: Reading something, understanding the gist and the meaning, and writing it down your understanding improves the understanding process
People who take hand-written notes better understand the material, when compared to people who are taking notes with a computer: I want to read the study on this. The author’s reasoning is that writing is slower than typing and hence we prepare ourselves to cut through the peripheries of the speaker’s arguments and get to the gist as quickly as you can and in as few words as you can. Everything else that’s part of the understanding process has to happen in your mind before you write down your understanding.
- I took hand-written notes for nearly 5 years in college. My understanding wasn’t really improved a lot because mostly I was simply copying what was being written on the board into my notebook. I was postponing the understanding to the 2 weeks before the test.
Once a piece is written down, what you meant is irrelevant. The only thing that matters is what you wrote down. Any piece disconnected from it’s author has the same goal: to make a convincing argument. And the author gives a clear description of a convincing argument:

The criteria for a convincing argument are always the same, regardless of who the author is or the status of the publisher: They have to be coherent and based on facts. Truth does not belong to anyone; it is the outcome of the scientific exchange of written ideas. This is why the presentation and the production of knowledge cannot be separated, but are rather two sides of the same coin (Peters and Schäfer 2006, 9). If writing is the medium of research and studying nothing else than research, then there is no reason not to work as if nothing else counts than writing.

Categorizing the information that is being fed into students in colleges into neat topics and grouping everything together is detrimentary to the student: Frequent context switching, using tests as a means to show students what they need to learn instead of using them to simply ensure that they are learning what they should have, allowing students to decide their curriculum autonomously
Avoiding feedback in the short term is disastrous: It leads to feeling smarter in the present, instead of becoming smarter in the future
Goal: Developing new ideas; connecting a variety of pieces together; answering the questions you have concretely; writing down your new idea because no other book has exactly what you are looking for.

P.S In a crazy connection, the author says that the book about the rise of the shipping container, The Box, that I read a few months ago was one of the reasons he realized that simple ideas find it especially hard to become popular and widely adopted. I read The Box because of my interest in shipping containers and the logistics world in general only a few months ago.

Tools

I have started looking into Org roam in the past couple days. With my recent switch to start using Emacs more in my daily workflow, I think integrating Org roam and using it to take notes should be easy to begin. I want to write notes about some of the topics that I have read about in the past few months (and heard podcasts about) that are connected to each other:

The book advises you to simply start writing notes for new literature that you read and forget about everything that you read until now. Some of the books I read in the past couple years were extremely important though, so I am definitely going to put my reading on hold for a week or so and actually write these notes down and get a good workflow going with my slipbox and reference system.

Using Emacs - A trial

2020-07-08T00:00:00+00:00

I have been used Vim for a while now. In university, my Vim usage was mostly related to the work I did in student groups. This probably came out to about 20-25 hours a week, on average. I used it to write my bachelors and masters thesis in LaTeX, which was probably the smoothest workflow I could have had for writing something that was about 80 pages long. Once I started working, my usage increased to nearly 40-45 hours a week. I was writing notes and code in Vim every weekday.

With this increased usage, the problems with Vim really started coming to the surface.

Problems

An inability to understand codebases

I was reading a huge PHP code base at work. I resorted to IntelliJ for PHP because IntelliJ IDEA really understands PHP code well, I couldn’t see how I would move around an unfamiliar code base on an editor that did not support it well enough. I tried ctags frontends, but they were not sufficient because switching to the function getUser on the line $transaction->getUser() using ctags would not lead to the Transaction class inside models/, instead going to ACL::getUser !?! (I later discovered that it is possible to switch between tag locations after the first jump; still not really very syntax aware)

An inability to do things in the background

A lot of has been said about this. Neovim is one of the popular alternatives that fixes the problem. I switched over from Vim to Neovim in February. They are basically the same underlying program, so I didn’t notice a drastic change just from this switch.

Poor handling of nested lists within Markdown

I was writing all my notes in Markdown. This was working well. I had a file for each month and a heading for each day. Inside each heading, I started taking notes that were nested lists. This is a sample block that is typical of the notes I was taking: (made up)

- Meeting: Some topic 1
	- Feature 1: Are we doing this?
		- Pros
			- Good for cats
		- Cons
			- Bad for cat food
	- Resources
		- New redis cache
		- New microservice?
			- Seems excessive for a prototype
			- Put logic in service A?
				- Service A has too much logic
				- `internal/src/partner/logic.go` => Too much unrelated logic in this file, this has
				already caused a few incidents during release
			- Service B
				- sounds better
				- Traffic is high, no problems in the past few months
	- Technical discussion
		- Cache with TTL?
		```go
		// Write logic
		redis := newRedis()
		redis.Set(key, value, -1) // -1 => No expiry

		// Read logic
		value := redis.Get(key)
		json.Unmarshal(value, &t)
		if time.Now().Since(t.CachedAt) > 5 * time.Minute {
			resp, err := callExternalService()
			if err != nil {
				return t.Response
			}

			return resp
		}
		```
	- Resolution
		- Feature 1 in next sprint
		- Logic in service B for now, move to microservice if prototype is successful

After the meeting was complete, I would look through it again and edit it to fix grammar or indentation and remove redundancies, elaborate things that weren’t immediately clear, make sure I didn’t miss anything. After this editing step, I would instinctively use gqip to reformat everything and make it look pretty again. This step would reformat the code block as if it was plain text! This was clearly a problem. The editor didn’t understand that this text is special and shouldn’t be touched when reformatting the text around it.

As I started getting involved in more projects and my notes started becoming more complicated and the levels of nesting grew, this problem became worse.

Theory

Vim doesn’t understand the text that is being edited.

Think about any language that has a structure which allows identifiers to have the same name but have different meanings when used with different objects. Eg: PHP, Ruby (Inheritance, dynamic type), JavaScript (types, definitions not required), Markdown (freeform).

Editing these languages in Vim is hard because simple text processing tools are NOT enough. There has to be a layer on top of the text processing that sits in between the text and the text processor and tells the text processor how to deal with each line of the text. This layer has the logic that is required to recognize that a line within a codeblock in Markdown shouldn’t be touched or (the much more involved interference) that the getUser() function called on a Transaction object is in transaction.php, not in acl.php.

On the other hand, take an extremely well-formed language like Golang: It comes with an excellent set of tools that sit between the text and the editor.

Eg:

gofmt => Every time the user asks the editor to save a file, the editor sends the file to gofmt and saves the output instead!
gopls / guru => When the user taps C-], the editor passes the cursor location to gopls and loads the file and line, column number that gopls returns.

If there were such tools for PHP, I am sure that PHP editing would be just as efficient as Go.

This is not a novel theory, it has been put forth by other people, in other forms. I simply realized it all over again from my own experience.

P.S: Language Server Protocol is a great project that aims to solve all these issues! I want to try PHP with it’s language server in Vim and in Emacs. (Emacs with Company in the LSP mode is actually a very powerful LSP frontend)

Solution (trial)

Switching editors seemed like the easiest solution to this problem. I wouldn’t be able to switch editors quickly; I started looking for alternatives anyway. Emacs stood out as the overwhelmingly popular and well-loved alternative. There were three talks that convinced me to start this trial:

The common thread in all these talks was that the speakers had become frustrated with the tools they were using and had decided to switch to another tool. They were not making an impassioned argument for any one tool, they were talking about how inefficient it can be to use the wrong tool. I had reached a similar point in my experience.

I hope to get used to Emacs enough to be able to use it as my primary editor on local computers. For remote instances, I will stick to vi. My familiarity with the Vim keybindings is quite useful in this regard.

Notes and Review - Lean In (Sandberg)

2020-06-01T00:00:00+00:00

There are a million reviews of this book out there, I don’t think I have anything new to say about the book. I do want to look at how Sandberg approaches the root cause of the issue that she deals with in this book: not enough women in leadership roles.

She begins by dividing the barriers into two categories: Internal and External. This was a very useful framework to think about the issue and try to improve the situation by making structural changes. The book is full of anecdotes, her conversations, and her advice to several groups of people. It is also relentlessly researched, there are no assertions or gut feelings in this book, everything is based on numbers from studies. For me, this was a book both about the main issue and a book about careers and how to make decisions that will affect you a few years down the line.

Highlights

Two things I took away from this book that changed the way I think:

Opportunity cost: This quote really got me thinking.

There is always an opportunity cost, and I don’t know anyone who feels comfortable with all their decisions. As a result, we inadvertently hold that discomfort against those who remind us of the path not taken.

Looking at the people I resented and have resented in the past, I can see them enter this criteria almost all the time.

Structural advantages: There are structural advantages that are not explicitly enforced, but do exist and favor the in-group, at the cost of the out-group.

The example that Sandberg talks about around the beginning of the book:

more men in leadership roles
men prefer working with other men
more men get promoted

Women pay the price for this structural advantage that men have. They don’t get promotions despite appearing to have the same opportunities.

Notes

Some of the other things that I have heard in passing but did not understand completely before I read this book:

Success and likeability: Positively correlated for men, negatively correlated for women. This was counter intuitive for me as I tend to like the people around me and am ambivalent about the people way above (2 or 3 levels up the report line) “Women who are liked are perceived as more nice than competent, but men who are liked are perceived as both nice and competent”
Helping coworkers: Women pay a bigger price for not helping their coworkers than men who decline to help others
- “Pronouns matter: Women should use the pronoun we instead of I” => This is similar to one of the observations that Ezra Klein made about Hillary Clinton when she said that one of the things that she had to learn and keep track of consciously was how she was taking credit for her accomplishments and how she taught herself to say I on the campaign trail instead of saying we.
Picking a new job: The only thing that matters when picking a job is growth. If a company is growing fast, there will be a lot of things to do and not enough people to do them. This pushes you into doing things that you would not consider a conventional part of your role. If a company is stagnant, there are a lot of people not doing anything.
- This was articulate and the first time I have heard it phrased like that. I am definitely going to use this in my own career.
Applying for positions: Women apply for jobs only when they meet 100% of the criteria, whereas men apply if they meet 60% or above. This is another structural area that comes back later in the book. She points to a doctor at a hospital who found out that his female students won’t raise their hand as often as his male students. So, he goes about fixing it by getting rid of open ended questions and instead pin-pointing people and asking them to answer.
Children: Don’t plan too far into the future; leaving a baby at home and returning to work will be very hard and the only thing that can make it worth your while is if you have a job that you are excited about.
Having it all: Perfection at work and home is impossible beyond a certain point. You must re-prioritize and decide what tasks can’t be compromised on. For people who are at the top of their fields (eg: Yale Law students), finding a mix at work and home is especially fraught with problems because they are used to demanding the very best from themselves and when this becomes impossible, they try to compensate by sacrificing sleep, etc.
Nouns and adjectives: Whoever has more power takes over the noun: (engineer, female engineer), (nurse, male nurse)
Lockjam: Talking about gender at work is hard, awkward and dangerous for managers and employees. There are legal implications that most people and HR departments would prefer to sidestep rather than wade through. Once we recognize what is wrong, we can not help but work to change it.
Gender: When asked to pick someone to collaborate with on a task, most people pick someone from the same gender! Organizations have to work explicitly to fix this skew by pushing people who don’t work with each other together for made-up reasons (eg: team building)
- This was a convincing reason for the “team building activities” that are in vogue lately

Quotes

To this day, I’m embarrassed that I didn’t realize that pregnant women needed reserved parking until I experienced my own aching feet. As one of Google’s most senior women, didn’t I have a special responsibility to think of this? But like Sergey, it had never occurred to me. The other pregnant women must have suffered in silence, not wanting to ask for special treatment. Or maybe they lacked the confidence or seniority to demand that the problem be fixed. Having one pregnant woman at the top—even one who looked like a whale—made the difference.

I don’t remember thinking about my future career differently from the male students. I also don’t remember any conversations about someday balancing work and children. My friends and I assumed that we would have both. Men and women competed openly and aggressively with one another in classes, activities, and job interviews. Just two generations removed from my grandmother, the playing field seemed to be level.

This experiment supports what research has already clearly shown: success and likeability are positively correlated for men and negatively correlated for women.3 When a man is successful, he is liked by both men and women. When a woman is successful, people of both genders like her less. This truth is both shocking and unsurprising: shocking because no one would ever admit to stereotyping on the basis of gender and unsurprising because clearly we do.

Then he explained that only one criterion mattered when picking a job—fast growth. When companies grow quickly, there are more things to do than there are people to do them. When companies grow more slowly or stop growing, there is less to do and too many people to not be doing them. Politics and stagnation set in, and everyone falters.

One thing that helps is to remember that feedback, like truth, is not absolute. Feedback is an opinion, grounded in observations and experiences, which allows us to know what impression we make on others. The information is revealing and potentially uncomfortable, which is why all of us would rather offer feedback to those who welcome it. If I make an observation or recommendation and someone reacts badly—or even just visibly tenses up—I quickly learn to save my comments for things that really matter.

But even if mothers are more naturally inclined toward nurturing, fathers can match that skill with knowledge and effort. If women want to succeed more at work and if men want to succeed more at home, these expectations have to be challenged. As Gloria Steinem once observed, “It’s not about biology, but about consciousness.”9 We overcome biology with consciousness in other areas.

True partnership in our homes does more than just benefit couples today; it also sets the stage for the next generation. The workplace has evolved more than the home in part because we enter it as adults, so each generation experiences a new dynamic. But the homes we create tend to be more rooted in our childhoods. My generation grew up watching our mothers do the child care and housework while our fathers earned the wages.

I started noticing how often employees were judged not by their objective performance, but by the subjective standard of how well they fit in.

Dr. John Probasco of the Johns Hopkins University School of Medicine told me that my story about women being more reluctant than men to raise their hands rang true for him so he decided to do away with the old hand-raising system during rounds. Instead, he started calling on male and female students evenly. He quickly realized that the women knew the answers just as well—or even better—than the men. In one day he increased female participation. By making one small change to his behavior, he changed a much larger dynamic.

Without calling for major overhauls, they tackled the soft stuff—small adjustments students could make immediately, like paying more attention to the language they used in class. They laid out a new, communal definition of leadership: “Leadership is about making others better as a result of your presence and making sure that impact lasts in your absence.”

There is always an opportunity cost, and I don’t know any woman who feels comfortable with all her decisions. As a result, we inadvertently hold that discomfort against those who remind us of the path not taken.

Notes and Review - The Odyssey (Homer) (Wilson translation)

2020-05-06T00:00:00+00:00

Review

I heard about this new translation on the Ezra Klein show. Madeline Miller made a very convincing pitch for why this book is relevant today, and why everyone should read this book; especially, the new translation by Emily Wilson. I was looking for something complex to read, that would keep me occupied for the 5 day extended vacation from May 2nd to May 7th (here in Japan). That was my main reason for picking this book up.

The story was simple, it was told at a beautiful, exciting clip. The story moves forward with this incredible, hard-to-believe speed. I am glad I read this book!

The story is fairly simple: Odysseus goes to Troy to fight alongside the Greeks against the Trojans. He’s cunning, and extremely shady; he comes up with several clever, “fox-like” plots throughout the story. Every time that he is despairing and cornered somewhere, he comes up with some kind of way to trick the person who has captured him and get out and get back to Ithaca, the place where he was king before he left for Troy.

At home, his wife Penelope and his son Telemachus are waiting for him. Well, sort-of. Telemachus is a child (although he is 20) and doesn’t really have much of a spine. Penelope definitely wants him to return, but it also feels like there are moments when she does not really want him to return. She’s never really happy throughout the book; even when he comes back and they are re-united, he tests him and then is happy that Odysseus is back; she is very mysterious and does not clarify her position to anyone. And finally, the villains of the story are the suitors: A group of obnoxious young men who are trying to win Penelope’s hand in marriage. They have somehow entered Odysseus’ house and have started having daily feasts there (???) The concept of someone entering your house just because you are not home and eating from your larder and having a great time in your absense is hard to comprehend. Perhaps it was something that was quite common in Ancient Greece? Exactly how they gained access, and established themself in Odysseus’ house is never made clear. They are there at the beginning of the book, and they are an obnoxious bunch.

Odysseus is also very popular with the ladies. Every female character he meets in the story immediately falls for him and wants to keep him tied up wherever they live. Calypso is the goddess who has a lot of power and tries to take his freedom and give him immortality instead; Odysseus is not down. Circe hangs out with him for about a year; then he decides that he wants to leave. Why he didn’t decide to leave early, if he really wanted to see his wife and son, is never explained.

Dichotomies

This book has a lot of dichotomies. Odysseus wants to get back home, but he’s always having a good time wherever he lands. He shows urgency at some points in the story, but the 1 year he spends with Circe “in her bed” is pretty hard to explain away.

Penelope wants her husband to come back home, she wants Telemachus to be safe and to mature into an adult. She doesn’t take any steps towards either end. She is very good at weaving apparently, so she is weaving in her room and weeping into her bed. No concrete steps though.

Telemachus wants to become a man, but he’s the most spineless character in the whole book. Every time that someone wants him to do something or decide something, he simply defers to the other person. The most glaring example of this that frustrated me was the one in which Odysseus has finally vanquished all the suitors and asks him to suggest some kind of way to avoid a confrontation with the suitors’ families. Telemachus replies promptly:

Telemachus said warily, “You have to work it out. They say you have the finest mind in all the world, no mortal man can rival you in cleverness”

Odysseus was not fishing for compliments man.

And finally, the dichotomy when describing city sackers and pirates.

Strangers, who are you? Where did you sail from? Are you on business, or just scouting round like pirates on the sea, who risk their lives to ravage foreign homes?

This was a pretty jarring line for me. “Risking your lives” is something that we always associate with something noble: like joining the army or becoming a doctor. “Ravaging foreign homes” is obviously very bad and not noble at all. To put both of those in the same line and frame a question is very hard to digest.

Translation

This book isn’t very long. It’s written in “iambic pentameter” and you can recognize the rhyme in most places, but it doesn’t read as a poem. It read likes prose. It’s extremely fast-paced. That was the highlight for me. I was able to finish reading this in about 3 days. I was reading slowly and taking notes and making sure that I wasn’t missing anything important. Even still, the pace was fast enough that I often ended up reading for 2 hours without noticing the time or page numbers.

I found Madeline Miller’s pitch for the book on the Ezra Klein show podcast and her review of the book very useful to understand the goal behind this translation.

One of the things that Homer has in the original is this incredible forward motion. It’s an exciting, exciting read. Wilson wanted to keep that galloping speed. – Madeline Miller (10:15, Ezra Klein Show, 2020-04-23) – Podcast

Notes

But of course, the English of the nineteenth or early twentieth century is no closer to Homeric Greek than the language of today. The use of a noncolloquial or archaizing linguistic register can blind readers to the real, inevitable, and vast gap between the Greek original and any modern translation.

– From the translator’s note. Wilson gets it absolutely right! I have always wondered why we must use antiquated English in our translations; she does a great job of explaining why that’s not useful and serves only to drive people away from reading literature that they would find interesting.

There are some really good portions in the poem. I really like the ones in which the characters are talking about their hatred for one another, or how something that the other character just said was so stupid that the person shouldn’t even consider talking ever again.

We suitors will keep eating up your wealth, and livelihood, as long as she pursues this plan the gods have put inside her heart

– The impunity in this statement from Antinous (probably) is incredible. How can you even say that, man!

Strangers, who are you? Where did you sail from? Are you on business, or just scouting round like pirates on the sea, who risk their lives to ravage foreign homes?

Do not be enraged at me, great goddess. You are quite right. I know my modest wife Penelope could never match your beauty. She is a human; you are deathless, ageless. But even so, I want to go back home, and every day I hope that day will come. If some god strikes me on the wine-dark sea, I will endure it. By now I am used to suffering – I have gone through so much, at sea and in the war. Let this come too.

As you know, divine Calypso held me in her cave, wanting to marry me; and likewise Circe, the trickster, trapped me, and she wanted me to be her husband. But she never swayed my heart, since when a man is far from home, living abroad, there is no sweeter thing than his own native land and family.

– This line was an articulate description of how I have felt every time I have moved away from a place I lived in for a long time. (moving to a new city when I was 10, moving to college when I was 17, moving to Japan when I was 22)

So you must never treat your wife too well. Do not let her know everything you know. Tell her some things, hide others. But your wife will not kill you, Odysseus. The wise Penelope is much too sensible to do such things. Your bride was very young when we went off to war. She had a baby still at her breats, who must be now a man. he will be glad when you come home and see him, and he will throw his arms around his father. That is how things should go. My wife prevented my eager eyes from gazing at my son. She killed me first.

– This is Agamemnon giving advise to Odysseus. It’s a strange thing to say; he was burned pretty bad by his wife, but still.

Scowling at him, Odysseus said, “Fool! I did not do you wrong or speak against you. I am not jealous of another beggar receiving gifts, however much he gets. This doorway can accommodate us both. Do not hog all the wealth; it is not yours.20 You seem to be a homeless man, like me. Gods give all mortal blessings. Do not stir me to fight or lose my temper. I am old but I will crack your ribs and smash your face to bloody pulp—then I will have a day of peace tomorrow; you will not return here to the palace of Odysseus.”

Fighting words; “I will crack your ribs and smash your face to bloody pulp” => this is so much better than an action scene.

Notes and Review - A Game As Old As Empire (Hiatt)

2020-04-19T00:00:00+00:00

Review

This book is everything that it claims to be on the book jacket. It “uncovers the inner workings of the institutions behind these economic manipulations”. In particular, it looks at some of the incredibly global institutions that are name dropped in a lot of contexts: World Bank, International Monetary Fund and the World Trade Organization. There are several stories here about loans that were given to countries which were supposed to build schools or upgrade the city hall building of a bustling city or one of a myriad of other reasons but never did that or help the people the money was supposed to help in any way.

I must admit that it was a pretty shocking revelation at several points. In particular, we get stories from the people in the field, the rank and file of organizations like the World Bank who are going abroad to assess if a given loan should be sanctioned or a banker who used to work in an island that was being used for offshore banking.

They tell stories about how their job affected the places that they were working in or the places they went to in Africa (this book focuses a lot on some countries in Sub-Saharan Africa) or about the ways in which the management and people above them were basically apathetic to the consequences of the things that their jobs were enabling. These stories are all backed up with actual data from reports or news articles which show the outcomes of the things that are discussed.

I particularly liked that the book doesn’t work as a list of testimonials where the reader is simply supposed to believe the people who are telling the story and the editor and not really question where they got their data. This was a refreshing non-fiction book about global economics, in that sense.

I learnt a lot of things about the global economy. The chapter on offshore banking in Jersey, Englang, the chapter on exploitation of the Iraqi people and government in the name of “Production Sharing Agreements”, the chapter about the strange policies that were implemented in Philippines despite continued realizations at different levels of the heirarchy that whatever they were doing was simply not working.

The most important insights for me came in the 11th Chapter by James S. Henry, The Mirage of Debt Relief. This chapter really shook me and my assumptions about national debt. I didn’t know much about national debt before I started reading the book, but my belief was that having high foreign debt was quite common in most economies and that it wasn’t really something to think much about because “banks and governments don’t default on loans”! This assumption of mine was completely blown out of the water by the things that Henry shows in this chapter.

On the Debt/Capital Flight cycle

Debt goes to the Finance ministry of a government with no oversight or accountability to the tax payer.

This capital is either wasted on projects that are intentionally priced above the market rate or is fleeced by the people inside the government or companies with close ties to people high up in the government who are supposedly working on “development projects”
The fleeced capital is moved out of the country through a web of offshore investments contributing to capital flight. This is powered by clever, high priced lawyers and bankers who are working on islands like Jersey, England (a previous chapter)
The country is left holding the bag: An unproductive loan that taxpayers have to pay interest on as debt service, each year. The principal will almost certainly not be paid for several years.

The existence of this cycle might appear to be semi-obvious if you have followed the trajectory of some countries and their economies: They get huge loans from the World Bank or from a group of foreign lenders, but the money never ends up making any difference. Eventually, the country gets a follow-up loan or everyone gives up on the country’s economy. Corruption and transparency issues are pegged as the root cause of this problem.

Here’s the sitch: The debt-based development model doesn’t work when it comes to low-middle income countries where the government is a weak institution and almost always over-run by corruption.

The World Bank model of neo-liberalism peddles the “free-market” as the silver bullet which is going to solve all the problems that plague the lower income economies of the world. But the problem with this model is that it is focused on lending to the government, reducing the government’s role in the economy, bringing in private players for local services and foreign investment for new industries, effectively reducing the government to a license-issuing institution with no real power to set policy or even the minimum wage that should be paid to citizens or even control the amount of local labor that must be used for a given project.

In search of the free-market, the government has just accepted World Bank advise and reduced it’s own role and made itself weaker. This makes it even more simpler for corrupt individuals to take over the government and fleece future and past loans even more efficiently! This kicks off the cycle which leads to a small elite who control the government, are extremely rich, keep their assets safely in First World economies in offshore investments, thus ensuring that anyone that the loan was originally meant for will never benefit from it.

Export Credit Agencies

The list of projects that were funded by ECAs from across the First World is quite long and each project has it’s own problems. As the author of this chapter (chapter 10), Bruce Rich, introduces a lot of issues with ECAs: Lack of oversight, lack of accountability to the tax payer even though they can enjoy the benefits of tax money and the government’s clout, an impenetrable curtain of secrecy that ensures that they don’t even have to publish the list of projects that they approved and funded! The author uses a phrase that can be used in several situations including while describing the development that was supposed to come from the huge loans that have been sanctioned over the past few years to the lower income countries:

perpetually around the corner

There are some important numbers and a little bit of discussion about why China, India and Korea had some structural, geographical and political advantages and thus were able to mostly ignore advise from the IMF and outright reject the neo-liberalism that is being peddled as the “road to development”.

I could not find anything problematic about the book. The sources are extensive and most of the facts that are not just personal experiences are backed up with official sources. My reading list after reading this book has a couple of papers about ECAs and how economists in the First World view them. The one thing that might threw me at first was how old all the data was. Reading the introduction gives some background on this, it appears as if this book has been around for a while but wasn’t published by anyone because no one wanted to touch a book that was flaty critical about the global economic system and what had basically become a cartel of the richest countries bending loans and lower income countries to their will.

Quotes

China and India alone account for about $500 billion of this developing country “present value debt.” Both countries have been careful about foreign borrowing, and they have also largely ignored IMF / World Bank policy advice. The result is that their foreign debt burdens are small relative to national income. Both countries – partly because they refuse the follow orthodox neoliberal policies – now have high growth economies and large stockpiles of foreign reserves.

– p.222

The fundamental problem, glossed over by some debt-relief campaigners and conventional “end poverty now” economists, is that comabting poverty is not just a question of providing malaria nets, vaccinces, and drinking water, or incremental increases in education, capital, technology and aid. Ultimately, as China’s example shows, long-term poverty reductions requires the promotion of deep-seated structural change. This implies the redistribution of social assets like land, education, technology, and political power. These are concepts that BWI [Bretton Woods Institutions] technocrats may never understand – or may recoil from in horro. But they are the root of every major development success story that we know.

– p.254

Notes and Review - Sapiens (Harari)

2020-04-12T00:00:00+00:00

I have been hearing about Sapiens since April 2018. A friend in university read that book and told me that it was really good and that it changed the way they thought about a lot of things. I wanted to read the book but it was never very high on my list. This past Friday, I read a profile of the author, Yuval Noah Harari in the New Yorker. This profile intrigued me because it told the story of a university professor and a historian and even a philosopher who had some very strange principles and said things that I had honestly never heard before. These two quotes sealed the deal for me:

In “Sapiens,” Harari writes in detail about a meeting in the desert between Apollo 11 astronauts and a Native American who dictated a message for them to take to the moon. The message, when later translated, was “They have come to steal your lands.” Harari’s text acknowledges that the story might be a “legend.”

“I don’t know if it’s a true story,” Harari told me. “It doesn’t matter–it’s a good story.” He rethought this. “It matters how you present it to the readers. I think I took care to make sure that at least intelligent readers will understand that it maybe didn’t happen.” (The story has been traced to a Johnny Carson monologue.)

I wanted to find out if I was an intelligent reader. And this one:

Later this year, in Israel, Harari plans to have a private conversation with Peterson. Harari said of Peterson’s representatives, “They offered to do a public debate. And we said that we don’t want to, because there is a danger that it will just be mud wrestling.”

I don’t know much about Jordan Peterson but I have seen him talk about some things when he was on the H3H3 podcast and the one feeling I didn’t get from him was that he was someone who would get into mud wrestling on a public debate. (Especially because they talk about his appearance on an English talk show and how the anchor tries to gotcha him every single time but he is extremely careful with his words and can effortlessly trace back to exactly what he said, word for word, and explain to her why she is portraying things not as they are, but as she wants them to be.

Now, I absolutely had to drop everything else and read Sapiens. I did just that.

Review

The book is divided into three parts.

Summary

TL;DR Don’t read Sapiens. Read Lever of Riches (Mokyr) to get a detailed account of the last 2000 years of technological progress and Factfulness (Rosling) to learn about the social advancements that we have made since ancient times and feel good about progress.

There aren’t a lot of original insights in this book, there’s a lot of speculation without any backing in facts or in arguments. The prose isn’t concrete enough to be history and isn’t abstract or argumentative enough to be philosophy. It is extremely easy to read and that is concerning to me because of the purported subject of the book: “History of Humankind”.

Most people who read this review might disagree with me or might think that this review is too harsh. I am confounded by the fact that so many people read this book and found it to be useful. I want to understand how that could happen. Perhaps, we can discuss why you thought it was a good book and what insights were in it that I didn’t pick up on.

Part 1

The first part covers human “history” leading upto about 12,000 years ago. The main event that it talks about is the Cognitive revolution about 70,000 years ago at which point in time there were several species of the genus Homo on the planet and the sapiens species was the first to start thinking. I have put the word history in quotation marks because most of it is made up. There are no written records (there wasn’t even writing yet) and there’s the flimsiest basis for most of the claims made in this part if you consider archaelogical findings. The ethos of this part seems to be “If you don’t know what happened, make something up”

One particular passage sums up this part nicely:

Tolerance is not a Sapiens trademark. In modern times, a small difference in skin color, dialect, or religion has been enough to prompt one group of Sapiens to set about exterminating another group. Would Ancient Sapiens have been any more tolerant towards an entirely different human species? It may well be that when Sapiens encountered Neanderthals, the result was the first and most significant ethnic-cleansing campaign in history.

– p.18

This passage comes about 3 lines after the author admits that there are two possibilites that expert have admitted: Sapiens exterminated Neanderthals either intentionally or by simply being better at foraging and hunting everything before Neanderthals could do anything OR Sapiens and Neanderthals intermixed and there was no malicious extermination. You might have seen what path the author chose. And perhaps you see why he chose that path. This passage is particularly hard to look past because it’s the equivalent of something that talking heads say on TV to get next day’s sound bite; it is sensational.

Part 1 is rife with such examples. Another classic:

We did not domesticate wheat. It domesticated us. Domesticate comes from the Latin domus, which means “house”. Who’s the one living in a house? Not the wheat. It’s the Sapiens.

– p.81

And one last one:

Since we enjoy affluence and security, and since our affluence and security are built on foundations laid by the Agricultural Revolution, we assume that the Agricultural Revolution was a wonderful improvement. Yet it is wrong to judge thousands of years of history from the perspective of today.

– p.83

Here’s the thing: what he is talking about is not wrong. No one can say that it is, because we don’t know what is right. We have absolutely no idea. No one does because there are no records from this period of time. As there are no records, one can say almost anything they want to about this period without caussing much ire among any group of scholars. In my opinion, one should not speculate about such things. Especially when the speculation is that it was a tragic mis-step to move from foraging to agriculture and the implication is that even though we have some comforts today the path we took to this state was not a good one.

I know that books about a particular philosophy make such speculations. That is completely acceptable as the author is trying to put forward an argument for why things are the way they are. This is not philosophy. This book is not being marketed as philosophy. The back cover of the book claims that it is related to “Science” and nowhere does it indicate that everything in this book might not be factual or even based on any evidence.

Part 2

This is the smallest part of the book. It talks about the effects of the Agricultural Revolution, about 12,000 years ago. Here, we slowly lead up to the invention of writing. Things start to get more concrete and the author moves away from using probably and perhaps. There are fewer over-simplifications in this part, but there is still a significant amount of “dumbing down” of complicated topics for the benefit of the reader.

I didn’t really get any novel insights from this part either. Most of the stuff being said is either mundane and known to everyone or hyperbole. I have tried to find some original insight that I can highlight here but I am struggling to even as I flip through the chapters to find something. This is a very good example of the kind of hyperbole: The author’s argument is that moving from foraging to agriculture was a tragic mis-step, people are supposedly working more hours, dependent on nature’s whims hence making them more anxious and are getting less nutrituous food in return. The only good point is that the food comes in abundance and one doesn’t have to live hand-to-mouth which leads to an explosion in population.

When discussing why such a mis-step was allowed to stand by thinking populations and evolution, on a macro level, alike, here’s what is said:

The above scenario explains the Agricultural Revolution as a mis-calculation. It’s very plausible. History is full for far more idiotic mis-calculations.

– p.89

There’s no explanation about what those other idiotic mis-calculations were; also, the very next paragraph explains another possibility which has written evidence.

Part 3

This is the longest part and it’s about the unification of humanity into a single world. For the first time in the book, there are some original insights. Even though they are framed as bite-sized couplets that would look very good with a contemplative image as background, they are insights nonetheless. here’s the first one:

Photo by Christian Kielberg on Unsplash

I didn’t have time to make similar images for the other insights, but there are some good ones. One particular story that I liked was about how James Cook’s expedition out to Australia was actually as part of an expedition that the British government had set up asking scientists to go to different places and get some readings which would enable them to calculate the distance between the Earth and the Sun accurately.

Another one was about how America was named after Amerigo even though the only thing the Italian Amerigo did was say that the landmass that Columbus found were part of an entirely unknown continent. This is a great piece of trivia.

There are more such pieces of trivia. But the major questions are left unanswered and there aren’t even any solid serious arguments put forth about them. For eg, the author started to make me uncomfortable for a few paragraphs by saying that Communism and even Nazism were religions!!! They were not mere ideologies that one could choose to believe at one point and not believe when they did not feel like it, they were religions that one must convert to and can’t leave without an official conversion, and so on. But after these few paragraphs, he immediately backs off and doesn’t flesh it out any further. I don’t think I would have the stomach to read about how Nazism is a religion (I recently finished reading Diary of a Man in Despair) but for Communism, it would have been an interesting argument to reason about. But the book disappoints by not taking it any further. In fact, the author even says that it’s not of any import what you call it:

If it makes you feel better, you can go on calling Communism an ideology and rather than a religion. It makes no difference.

– p.229

If this was indeed the case, then why make the initial argument and say that they are religions?

Another question that is asked is about why the Chinese or the Mughals or the Indians weren’t interested in going over to America and conquering those lands for themselves despite finding out about European conquests on this new continent almost immediately and having the ocean navigation technology to get there. Readers of this blog might notice that this is extremely similar to the questions posed by Mokyr in Lever of Riches. The difference being that Mokyr presents the comparison in extreme detail and puts forth several arguments for why he thinks this could have been. Nothing of this sort can be found in Sapiens. The question is posed, and promptly forgotten about.

The last three chapters of the book are apart on their own. The book slowly turns into a social commentary on our connected way of life, consumerism, people buying things they don’t need, happiness being connected to what one feels rather than what one has, “pauper can be happy if they have a loving family but a lonely billionair can’t” and so on. Once again, there’s nothing particularly revolutionary about anything being said. I felt like I was reading curated Medium articles about capitalism, happiness, consumerism, the evils of shopping malls, global trade connecting nations etc.

There are some hard to believe assertions:

Never before has peace been so prevalent that people could not even imagine war

– p.344

The author says this in reference to the post-WW2 period from 1945 to now. As far as I can remember, most of the cold war was anxiety filled about an actual war, there was an actual war in Vietnam, then several wars in the middle east. In India and across the world, there was some terrible terrorism in the period between 2000-2012. So, I would say that “people could not even imagine war” is an exaggeration. There’s also no mention about the most peaceful period in history which one might compare our own times against.

Some quotes from the book

An example of sentences that start with probably and how detailed they can be: (emphasis on probably mine)

Nomadic bands that stalked wild sheep gradualy altered the constitutions of the herds on which they preyed. This process probably began with selevive hunting. Humans learnt that it was to their advantage to hunt only adult rams and old or sick sheep. They spared fertile females and young lambs in order to safeguard the long term vitality of the local herd. The second step might have been to actively defend the herd againt predators. Driving away lions, wolves and rival human bands. The band might next have coralled the herd into a narrow gorge, in order to better control and defend it. Finally, people began to make a more careful selection among the sheep, in order to tailer them to human needs. The most aggresive rams, the ones that showed the greatest resistance to human control, were slaughtered first. So were the skinniest females and the most curious. With each passing generation, the sheep became fatter more submissive and less curious. Voila! Mary had a little lamb and everywhere that Mary went the lamb was sure to go.

– p.92

An example of the arguments that the author decided to back out of instead of fleshing them out further:

If it makes you feel better, you can go on calling Communism an ideology and rather than a religion. It makes no difference.

– p.229

A crucial lack of explanation about what the theories are and why they aren’t convincinng:

Why did the Scientific Revolution begin there of all places, and not in China or India? Why did it begin at the midpoint of the second millenium rather than 2 centuries before or 3 centuries later? We don’t know. Scholars have proposed dozens of theories but none of them is convincing.

– p.244

For posterity, the arguments given in Lever of Riches (Mokyr) are convincing. The least the author could do was at least point the reader to another book which would summarize those theories.

P.S. I wrote about this on Twitter as I was reading the book:

Converting GNUCash's XML file to Ledger's DAT format

2020-04-05T00:00:00+00:00

Note: What follows is a meandering account of why I wrote a Perl script to convert GNUCash’s XML file to Ledger’s plaintext format. This script was the result of this process.

I have been using GNUCash to maintain my personal finances for about 4 years now, and especially since I started working, I have used a single GNUCash book for my expenses. Before that, I used to close my accounts every year or half year and start a new book. Apparently, this is not the recommended way to use GNUCash.

The more I have used GNUCash, the more I have become familiar with the data entry system. In general, I try to enter all my transactions by saving receipts and categorizing everything appropriately. If ever I have the feeling that a particular month or quarter I spent too much on a given category (bought too many clothes / too much electronics shopping) I pull up the Expense barchart report that GNUCash has in built support for. The default report is pretty good, and gives one a good idea of how they are doing with their money.

The Ideal Flow

Ledger came across my radar when I was researching for alternative tools that automate the entry process. In particular, the flow I had in mind was this:

Right after the transaction, enter the transaction in my phone into a note or send an email to myself
Some cronjob takes these emails and converts them into a file that GNUCash can parse
Import this file into GNUCash at the end of each week

This might have been too ambitious, because I couldn’t find one easy to import format for GNUCash. I wanted some convenience features too. The major one was “Transaction Aliases”:

CC F L RA 500 => Lunch at Restaurant A, 500 Yen, => Debit from credit card, Credit to Expenses:Food:Lunch

This was definitely not part of the MVP that I was envisioning, but I wanted to be able to build this at some point in time. I even dreamed up a fictional DML that would support both aliased transactions and new transactions. I am sad to say though that none of this ever became reality.

Well, what did become reality then?

A script that converts the GNUCash’s XML formatted file into the file format that Ledger accepts as input. (A text file which commonly has the extension .dat)

Ledger is a very interesting project. It attempts to be a complete accounting and finance monitoring tool (which can be used even by enterprises which have complex transactions) that’s based completely on a single text file.

There is a DML that defines transactions
There are no database type “IDs” that are attached to accounts or commodities
Every time you run Ledger, the whole file is parsed
Ledger doesn’t control the data entry. That is, you can enter data however you wish. You can export data from GNUCash to Ledger, or from a spreadsheet to Ledger, or write the Ledger entries by hand (very painful) or generate it from other structured format using a script

All Ledger cares about is that your DAT file should have a bunch of transactions, and these transactions must balance.

Now, if the DAT file has a set of transactions and they balance, then Ledger can generate a variety of reports for you. This is where things get interesting.

(A little bit of accounting knowledge from a complete beginner) There are three accounting reports that you will find on every single company’s quarterly earnings reports. Apparently, these reports are big deals and tell everything one needs to know about the “financials” of a company. I don’t know much about why or how to read one and learn new information. I did want to generate these three for my own personal finances though.

Ledger lets you do this extremely easily.

hledger -f simple.dat balancesheet

This prints a plaintext balancesheet. It lists all the assets that you have (bank balances, cash, stocks) and all your liabilities (credit card outstanding, loans). I found my balancesheet to be rather interesting. I didn’t learn anything particularly new, but it was a fun project.

Why Perl though?

Perl has a pretty good XML parsing library. I have also read a bunch of blog posts that say that Perl is the best language for massaging text and moving it from one format to another format. I wanted to use Perl for a real world project and see what advantages it had and whether I would be able to really use it on a daily basis without having to look at perldoc every 10 seconds.

I did’t really get too far with my actual Perl expertise, but I did learn a few great things about Perl itself. Such as the difference between scalars, arrays, hashes and references. And how Perl understands what context a variable is used in (i.e an array’s length is used if you compare it against 0 inside an if clause)

In conclusion though, I wanted to try a new scripting language and this seemed as good a project as any other out there.

More tools!

2020-02-29T00:00:00+00:00

Here’s a list of some of the repositories I created on Github recently:

Vim plugins
MarkdownLinks.vim	2020-02-02T03:54:14Z
PersistentScratch.vim	2020-01-02T09:56:07Z
Scripts in bash / perl / etc
gospec	2020-01-25T12:08:24Z
json-where	2019-09-23T07:45:08Z
gnucash-xml-to-ledger-dat	2020-02-11T11:35:33Z
launchdarkly-prereq-graph-gen	2019-06-01T08:02:52Z
Firefox add-ons
wealias-firefox-add-on	2019-02-05T13:49:12Z
open-circleci-workflows-firefox	2019-08-24T07:13:19Z
Misc
stern	2019-03-21T14:47:12Z

I realized that I have been building a lot of tools lately. In particular, I have gotten into writing Vim plugins and forking and editing the tools I use often so that they have the options I want, but will probably never be merged into the tool’s repository itself.

I generated this list by making a request to the Github API:

$ curl -s -G --data-urlencode "sort=newest" \
        "https://api.github.com/users/icyflame/repos" |\
        jq -r '.[] | [ .name, .created_at ] | @tsv' |\
        tsv-pretty

If you have been working on a lot of open source stuff lately, but can’t figure out what exactly it is you spend most of your time doing, I definitely recommend looking through Github’s API!

Vim Plugins

I have been a Vim user for a few years now and I have written extensively about it! The one thing I hadn’t done before was write plugins. So, I started looking into writing a plugin with the specific goal of writing a plugin that would store the currently open buffer in a file somewhere on my file-system. Where the file was stored wasn’t really important, it didn’t even matter what the name of the file was, all the mattered was that it was on my persistent storage somewhere. If I knew that the file was stored, it would allow me to get text (like deployment pipelines), store them on my file-system and go ahead and edit them directly without having to think about where I should store it or what I should name them.

It turns out that Vim’s plugin API is extremely easy to use. There are two particular things about it that I like a lot:

Standardized file-structure

$ tree --charset ascii
.
|-- autoload
|   `-- persistent_scratch.vim
|-- LICENSE
|-- plugin
|   `-- persistent_scratch.vim
`-- README.md

2 directories, 4 files

Most Vim plugins are very similar:

An autoload/plugin.vim file with all the keybindings and calls out to the functions which implement the plugin’s functionality
A plugin/plugin.vim file with the plugin’s functionality and some exposed functions which can be called by the keybindings / commands defined in the autoload file

This simple file structure makes it very simple to write a new plugin by looking at an existing plugin and trying to do the same things but for your use-case. My plugin reference was mtth/scratch.vim. My plugin was very similar to that plugin except for the part where the file is being stored on persistent storage.

Easy to test

VimScript is an interpreted language (I think?) and one can test plugins by simply symlinking the plugin’s development directory to a directory inside ~/.vim/bundle. After doing that, you only have to open Vim and start testing your commands / keybindings.

After PersistentScratch, I migrated MarkdownLinks out from my Vimrc out to it’s own plugin. MarkdownLinks is a plugin which adds a hyperlink to the word that your cursor is on or the highlighted text and adds the template [ around the word and the declaration of the link at the end of the file. I was surprised that something like this doesn’t already exist. I wonder how other Markdown writers deal with this.

Note: I actually started using Neovim instead of Vim recently. More on this in another blog post. Hint: The reasons were mostly because of my disillusionment with the BDFL model of governance in open source projects.

Bash functions + Utilities

json-where is something that I have been working on for a while now. It has been extremely useful to me whenever I am reading Kubernetes manifests. (Kubernetes manifests are notorious for containing extremely nested keys with vital information) It tells you the key given a location inside a JSON buffer. This works perfectly from inside Vim, where you can simply run a keybinding and get the path to the key your cursor is on.

gospec is the other thing that I have recently spent some time on. It’s a prettifier for the output produced by the go test command. I have been using it daily at work and it has reduced the time I spend using cumbersome tools (like ag + sort -r) when debugging tests for new code. It doesn’t do anything very complicated and the function itself fits within 100 lines of bash.

Misc

I have been writing some Perl recently. Perl is a surprising language as it has been around for a long time and it has remained mostly unchanged. The text handling capacities of bash are limited, and I use Gawk whenever I have to change streams of text, but sometimes even Gawk is not upto the task, especially when JSON is in the mix. One recent example was when I wanted to list all the Kubernetes pods and output the namespaces which had a particular environment variable set to a given value. I ended up writing a NodeJS script for this, but I definitely wished for something faster in the moment and I have been wondering if Perl could have been that language.

Perl’s is able to deal with HTTP requests, XML (some APIs are still XML), and JSON effortlessly. I want to start including it more in my daily scripting tasks. Something that I started using just yesterday:

echo -n "https://vendor.com" | perl -MURI::Escape -e "print uri_escape(<>)"

The 2 times I had to URL escape some text, I loaded up JSBin.com and entered it into the console. It took me a couple minutes to find the JS function which escapes text. So, yeah, this approach is way faster.

Work from Home

2020-02-29T00:00:00+00:00

Mercari announced on 18th February, 2020 that they will be implementing a work from home polcy for all employees starting the next day and continuing till the end of February. This past week the policy was extended one more week, until March 8th. More extensions are possibly around the corner.

So, I have been working from home since 19th February! I have a desk and a monitor at home. I brought my split keyboard (A Kinesis Freestyle 2) back home from the office. Setup-wise, I would say I have managed to re-create a very work-like environment. My house is a typical apartment in Tokyo with a floor space of 25 square meters (about 250 sq. feet). There’s a living space of about 150 sq.m and the rest is a kitchen, a bathroom and space for a refrigerator and a washing machine.

Work from home has been going fairly well, I believe. I was working on a project with about 4 other engineers. Understandably, communication has gotten slightly harder because we don’t sit 3 feet from each other 8 hours every day but fundamentally, we have a good idea of what we need to do and how everyone else is progressing with what they are doing. Often times when working on a big project that has several components with different people working on each component, there’s a lack of clarity about which piece is being built and how the pieces are dependent on each other. For backend development, this is perhaps even more of a problem because the way we break things down is generally by the various stages in a user’s usage of the project (the so-called user story) rather than the actual parts of the various repositories one would be changing. Most of the time, these things actually match up fairly easily and there’s no conflicts / wasted time because two people ended up writing the same code and one person happened to be done first.

Another part of the development process which I expected to take a hit from not working in an office was code review. This, I am happy to say, I have been wrong about. Code reviews have been coming in at the same pace, and there might even have been a slight decrease in the time between consecutive rounds of review because everyone is keeping fairly strict hours. While working at the office (with the privilege of having flexible hours) there were only a few hours of the day when everyone was at work at the same time and one could go through several cycles of review pretty quickly, while also addressing all the feedback that the reviewer had in the previous cycle. This guide on why code reviews should be done within a full working day has been very useful. It’s a pleasure working with developers who have read this / follow the advice in this instinctively.

Year in Review - 2019

2020-02-02T00:00:00+00:00

2019! It was my first full year at work, first full year outside India, the first time I spent a considerable amount of time learning and conversing in a foreign language. I looked back on the things I did and experienced in 2019 and wrote down the ones that stood out.

Travel

I went to Europe for the first time! It was also the first time I travelled outside India with my parents. We spent about a week in Italy! We went to Vatican City and stood in the plaza in front of St. Peter’s Basilica. I enjoyed the food and coffee that is especially in abundance in Italy, the amazing nature around Lake Como and the clean, crisp mid-August air. I also realized that most of the paintings in Rome were about war, strife, hunger and conflict.

I also went snowboarding for the first time. It was a back-breaking experience, I look forward to going once every year.

I visited Hiroshima and that was a very good trip. It’s a small city and so much of it is built around the A-Bomb from 74 years ago. It was an essential experience for me, to see the devastation that the bomb caused, to hear about the people it affected and to be in the city that I had read about all my life, without quite internalizing the impact.

Note to self: Arrive around noon, instead of after 8 pm at the country’s airport. Bring an empty water bottle and buy 2L bottles from a grocery store. Set a day aside to go to the various restaurants in whichever city you are staying in.

Books and Podcasts

I did not get to my goal of 25 books in 2019. I did read some very good books though. The highlights for me were:

Stranger in a Strange Land (Heinlein)
Eichmann in Jerusalem (Arendt)
The Box (Levinson)
The Lever of Riches (Mokyr)

I realized that I really like non-fiction written by economists. So, I am going to be reading more of that in the coming years.

On the podcast front, I really got into podcasts this year! I heard The Daily everyday, the whole year and there were some amazing episodes and series. Looking through the list of episodes on PocketCasts, I see some starred episodes:

The Jungle Prince series - November
The Origins of Boeing’s 737 Max Crisis - July - I heard this right before getting on a 14 hour flight to Munich. The management style’s direct impact on the factory floor was surprising when you hear about it from someone in middle management.
The President and the Publisher - February - “He just wanted his home town’s newspaper to do a good story about him” - Maybe it was an act, maybe it was real. Either way, the vulnerability and the detail with which this episode detailed Trump’s interview with NYT’s publisher and journalists was good.

Music

I started using Spotify Premium. I don’t say this lightly, it has been the best decision ever. The catalogue is un-ending, I have found some incredible music through Spotify’s recommendations. The best artists I discovered last year were:

Suzanne Vega
Cass McCombs
Elliot Smith
Metronomy (co-worker played The Bay at a work mixer!)
Jack White

I also saw Metronomy perform live in Shibuya. I went to a bunch of classical piano recitals at the Sumida Triphony Hall. And I saw Max Richter perform Memoryhouse and Blue Notebooks live!

Note: Visit the Ironbound market in Pennsylvania and Torbay in Devon, England. Use the Spotify API to backup the monthly playlists in a spreadsheet and store that in Dropbox.

TV and Movies

At long last, I got into The Office (US) and started watching it. I like the first four seasons, I am still ambivalent about the later seasons. I watched several episodes of Seinfeld and Modern Family repeatedly. Seinfeld still remains at the top of my list for comfort-TV!

I discovered Terrace House and watched all it’s existing seasons. It is a unique kind of unscripted show and probably one of the most useful resources in my efforts to learn Japanese.

I didn’t watch as many new movies as I would have liked to. Coffee in Berlin (2014) was a really good movie that I watched randomly on a flight. I can’t think of any other movie that I watched for the first time in 2019. A dismal showing.

Technology

I came across and started using a lot of stuff. The changes in my dotfiles over the year are especially indicative of this.

Started using prezto instead of zsh
Wrote the command prompt in prezto so that it would be exactly like ys in zsh. This took me a good 3 hours, but I think it was perhaps the best use of my time! I know exactly how to change every single character in the prompt now!
Configured Vim to work as I expected and wanted it to. I started using Slimux and UltiSnips among a lot of other plugins which were especially useful at work. I wrote my first Vim function and Vim plugin and realized the opportunities that I had to customize it further
Customized the statusline for Vim and Tmux. Tmux bindings to resize by entering the width and height
Re-mapped CAPS to ESC => This was a co-worker’s recommendation and it has been a game changer for me. My hands move less and it’s incredibly convenient because I never used the CAPS key before. On the flip side, I have gotten much slower at using someone else’s computer when they don’t have this mapping.

Apart from the configuration changes, I wrote a lot of bash and subsequently, I am actually okay at it now. I don’t have to look at the Bash Comparison operators and Bash if-else operators help page as often as I used to. I got slghtly better at awk and sed, but my proficiency in those hasn’t increased substantially.

Work

2019 was my first full year at work. I learned several new things about organizations and how they work. I learnt more about Search and Golang. I reviewed a lot of code and would like to think that I got better at reviewing and getting reviews, but that’s questionable at best. I got used to the development process at work and I got used to my role in it. I have only been at work a year now, so I can’t really say I gained a lot of insights about anything in particular.

I did get much better at explaining complex concepts to people in the same room and people halfway across the world by sharing my screens and preparing UML diagrams. I started writing basic PlantUML without looking at the PlantUML examples webpage. Still a long way to go though, my diagrams often miss boxes because I don’t know how to make them off the top of my head.

And finally, I got better at taking notes that will stay with me forever. I have a simple system of one file per month, I keep writing things in that file and keep everything in the same folder. Using ag, I can search for anything no matter when I wrote it and this has been helpful especially when I want to find out the reason for some of my choices, a few months down the line. I am writing everything in Markdown (including this post) and I am looking forward to building more tools around my workflow.

Notes and Review - The Trial (Kafka)

2020-01-18T00:00:00+00:00

The text cannot be altered, and the various opinions are often no more than an expression of despair over it.

This text perfectly describes this book. But the author put it at around the 90% mark, right when the book is about to end and you are starting to understand the kind of closure you will get from this book.

As far as disorienting books go, I have found those with non-linear timelines to be the most disorienting. In particular, you don’t know what’s going on, you have completely lost your place in the story of the book, you are invested in seeing the main character get out of whatever jam they are in but you have absolutely no clue where to go or which way will take the character forward and which will simply be a regression.

The Trial started (for me) with all of these confusions. K. was arrested, but he could keep working. The men who came to his house to tell him he was arrested were strange and didn’t behave as powerful men would have, K. is extremely apologetic to his neighbours and does some strange stuff: visiting a female neighbour at around 11 pm and so on. He is annoyed and confused, just as I was annoyed and confused at the author’s disinterest in telling me what was going on.

This feeling never quite goes away, throughout the book. At every step, you think this new character is probably not going to be very important, and they are not, but you get a lot of exposition about them and if it’s a female character, they are immediately taken by K.’s looks and fall in love with him. The author even addresses this himself while talking about how all the accused were attractive in one way or another:

this peculiarity of hers consists in this; Leni finds most of the accused attractive. She attaches herself to each of them, loves each of them, even seems to be loved by each of them; then she sometimes entertains me by telling me about them when I allow her to. I am not so astonished by all of this as you seem to be.

The other thing that Kafka does a disturbingly good job at is showing government offices, the people working in them and the way proceedings are handled. I had to visit the Regional Passport Office in Kolkata, India once. And the first trip that K. takes on a Sunday to the court sounded eerily famiiar because it was on a similar errand: I had no idea why I didn’t have my passport yet, I had to wait in line from about 10 am to 2 pm, there were a lot of people around me, the clerks in the office were the king-makers that day, the inability to see any of the senior officers in the office who would certainly understand my situation and finally, the breathlessness when you are just inside the door and are not yet used to the universal lack of ventilation.

As a result, the accused and his defence don’t have access even to the court records, and especially not to the indictment, and that means we generally don’t know - or at least not precisely - what the first documents need to be about, which means that if they do contain anything of relevance to the case it’s only by a lucky coincidence.

P.S. The second-last chapter in the cathedral leading up to the ending is weird. It has an anecdote about a doorman and it sounds vaguely similar to K.’s story and experience with the court, but not really.

Some quotes

he admits he doesn’t know the law and at the same time insists he’s innocent.

– When stated as literally as that, it seems rather stupid to ever say that one is innocent or that someone else is guilty because no one knows the law. When we say we’re innocent, I think we tend to use our value judgement of right and wrong and try to decide what we would classify what we did as; thinking about innocence as something determined by one’s value system and not by the written word of law.

He had, of course, begun work straight away and was nearly ready to submit the first documents. They would be very important because the first impression made by the defence will often determine the whole course of the proceedings. Unfortunately, though, he would still have to make it clear to K. that the first documents submitted are sometimes not even read by the court. They simply put them with the other documents and point out that, for the time being, questioning and observing the accused are much more important than anything written.

– Catch-22 style contradiction! LOVE IT

As a result, the accused and his defence don’t have access even to the court records, and especially not to the indictment, and that means we generally don’t know - or at least not precisely - what the first documents need to be about, which means that if they do contain anything of relevance to the case it’s only by a lucky coincidence.

“After a certain point in the proceedings,” said the lawyer quietly and calmly, “nothing new of any importance ever happens. So many litigants, at the same stage in their trials, have stood before me just like you are now and spoken in the same way.”

Notes and Review - The Lever of Riches (Mokyr)

2020-01-01T00:00:00+00:00

Summary

The Lever of Riches is a great book. I think I say that about a lot of books though. Lever of Riches is a 3-part book. In the first part, Mokyr presents a concise history of Technological progress starting in 500 BC and ending around \1915. In the second part, Mokyr compares the relative technological progress in three periods across time and space and the possible reasons (he touches on religion, culture, geography and national sentiment as possible reasons). In the last part, he draws an analogy between Biological Evolution and Technological progress.

I liked the first and second parts immensely. I didn’t find the last part (the analogy) very intersting or useful.

Some of the notes I made as I was reading this book.

Part 1

Water wheels are very important!
Compass - 1300 - A single self-contained unit; this made year-round navigation possible
Printing Press - Gutenberg - 1453 - In the 50 years following this invention, more books were produced as compared to the preceding 1000 years.
The educated class and the working class must interact closely and regularly. People in the educated class have the potential and opportunity to innovate and create new solutions whereas only people in the working class know what problems exist. If there is a disconnect between these two groups of people (as in Ancient Societies where children were educated and kept far from all proagmatic matters) innovation suffers.
Double Entry Book-keeping - 1350
Steam Engine - 1712 - 4.5% efficiency

In conjunction with other inventions, power technology created the gap between Europe and the rest of the world, a temporary dis-equilibrium that allowed the Europeans to establish global political and military domination.

Electric generator - 1870 - Stable, continuous current without over-heating
Progress in communication - Early 1800s, London -> Calcutta -> London postage might take as long as 2 years. 1914 -> Less than 2 weeks!
Bicycle - 1885 - The first means of transportation that was cheap and truly personal
Typewriter - 1874 - The Remington Company buys the patent for the typewriter
Automobile - 1876 - Otto creates the 4-stroke engine
Radio - 1906

Technological growth seems to be the only avenue of sustained growth

Part 2

Every invention has private and societal costs - Are both parties willing to bear this cost?
Inventors must be willing to bear risk to change the status quo. A lot of factors that inventors struggle with are permissive rather than causal
Technological progress is path dependent - Progress begets Progress
- Societies that accidentally bet on the slower horse might not realize how much potential other players in the race had! They might drop an idea simply because the first iteration they chose was incorrect
  - Camels in Middle East - 500 BC - This precluded the building of roads, but it also precluded the invention of rail roads!
Religion affects inventors
- The Christian attitude towards nature helps - The world was created by God for man and nature exists to be manipulated and enjoyed
- India’s entrenched caste system is the ultimate status-quo maintaining structure
  - “Hindi doctrine holds that castepromotion is possible through re-incarnation if one lives an appropriately resigned and obedient life. … Poverty was holy and action was vanity.”
  - “A fiendish failure proof system to ensure the status quo”
  - This discourages rebeliousness
Values - What do members of a society value?
- Ancient societies valued art and military accomplishments above wealth
- Do the educated work? Are the people who work educated?
Openness to new information
- Islamic beliefs started to converge to a point where it was believed that most of the information had already been discovered and that a modern Muslim need only accept and repeat

Part 3

Dynastic states are tolerant of non-conformist behaviour
Political unification doesn’t imply peace
There was no demand for European goods in China whereas Europeans were interested in a lot of things produced only in the Orient
A single ruler instead of several rulers like in Europe - A single ruler can stop any technology in it’s tracks with minimal effort. In Europe, inventors could fairly easily emigrate to a nearby nation and hence it was harder for rulers to suppress innovation for long
Technological progress went forward unabated in Britain because the rich and powerful held their wealth in Real Estate and Agriculture and these were not threatened by the progress in (say) the cotton industry
Britain’s decline can probably be attributed to a dislike of applied science

A gentleman only concerns himself with the lofty ideals of pure science, scientific research for it’s own sake

Review

Great summarization: What is this book about? I think that if you read the Introduction part of the book, you understand that and know exactly what’s about to come. So many books are free-wheeling narratives of the author’s research areas that it has few (if any) connections to the topic at hand once you are deep into the book. This book doesn’t lose the plot. It aims to provide one possible explanation for the apparent difference in quality of life between the West and most of Asia and Africa.

If the West is on teh whole comfortable, even opulent, compared to the appaling poverty still rampant in most of Asia and Africa, it is in large part thanks to its technology.

Excellent structure: I started noticing this in non-fiction books after the majorly disorienting narrative in The Box (Levinson). That was also a great book, but the fact that the narrative wasn’t chronological really messed up my comprehension of the book. Lever of Riches stays on track and moves linearly through the Ancient times, the Industrial Revolution to the late 20th century. There are no unnecessary breaks, few parallel narratives which are always geographically isolated.
Description and analysis of other prevalent theories: The author leans heavily on work done by others in his attempt to prove his own theory. And along this line, it was fairly clear to me, every step along the way that whatever the author was saying was thoroughly researched and that the author had seriously considered what other people believed and had not dismissed their points of view out of hand.
History of technological progress: The second part of the book is a concise history of technological progress. There are several diagrams and a lot of dates and names. I felt that the author did justice to this topic and I learnt a LOT just from this part of the book. If anything, I highly recommend this part of the book in isolation. A better book of technological history definitely exists out there and the author admits as much, but I felt that the detail Mokyr goes into is just right. (My opinion is bound to change when I read a more thorough history which touches on some of the things Mokyr skipped over)
Great comparisons: The comparisons that make up the third part of the book are definitely my highlight from this book. I liked the times and countries he compared because I have struggled in the past to understand the vast gap in the servies and quality of life in these places first hand, and this book helped me understand one probable cause for these differences.
Not crazy about the Evolution analogy: The last part of the book is an attempt by Mokyr to draw several parallels between biological evolution and technological progress. The analogy seems to hold in most cases and there are several holes, etc. In general though, I didn’t like this part of the book as much as I liked the preceding 2 parts.

Setting up Wireguard - Experience

2019-12-22T00:00:00+00:00

Note: There are several guides out there which have a set of steps and the commands to set-up Wireguard on a Linux computer. This post is written along those lines but takes a different approach - it focuses more on what one can learn about basic Linux networking by doing this setup themselves.

Premise

I started out with a fairly clear goal: Setup a VPN server inside a Digital Ocean droplet to forward all local traffic through, using wireguard.

I wanted to use Wireguard mainly because it was inside the kernel which had this subconscious implication that it would be blazing fast. Also, I have seen the video on Wireguard’s website which has Alice and Bob side by side and in about 7 commands the video shows how Bob: ping Alice starts working! This was fairly revoutionary to me because until now I had used two kinds of VPNs:

The free apps: This includes SurfEasy, which is how I used to use Spotify in India before they offered their service here, CloudFlare’s new app which is pretty easy to use but works only on my phone for the time being.
The enterprise kind: This is for work, and it is extremely painfree. It’s also fast enough that I haven’t noticed a big slow down that often. I don’t really know how it is all set-up inside, but like every company, there are some professional IT engineers who have set this up so that no one wastes any time trying to figure this out.

My current goal was to get something up and running which was easy to set-up, fast when I was using it and worked seemlessly on a Linux desktop. Wireguard seemed like the only choice.

Sidenote: What about Streisand?

If you are into self-setup VPN servers, you might have heard of Streisand. It’s a huge Ansible playbook which will take some input from the user and then setup a VPN server on the provider of your choice. The server that it sets up can be used as a Tor bridge, an OpenVPN server, a Wireguard server, an HTTP proxy server, SOCKS proxy or as an IPSec server. Overwhelmed?

If you aren’t overwhelmed yet, then let me tell you how the setup works: You run the setup script, you enter your provider of choice, what you want (automatic setup of all the provided components is encouraged, although you can choose to install only the components you want, if you are so inclined), and then Ansible starts it’s job. It will start running these playbooks on the VPS server.

Everything is pretty good until this point and at this point, the last time I was setting this up, I decided to leave my computer be and get some food. When I came back, I saw an error message that was waiting for me. I didn’t really get it, so I googled it found the problem, fixed it and ran the setup script again. Guess what? It started setting everything up from scratch. Yes. After seeing that, I gave up on Streisand.

Sidenote: What about a simple HTTP proxy?

The Squid HTTP Proxy is great. It has a simple configuration file format and it’s possible to run it using docker on any computer anywhere.

# Prepare the configuration file
docker run --rm sameersbn/squid \
    cat /etc/squid/squid.conf > squid.conf
# Edit the configuration file
# Start the proxy server using this configuration
docker run --name squid -d --restart=always \
    --publish 3128:3128 \
    -v "$PWD/squid.conf":/etc/squid/squid.conf \
    sameersbn/squid

If your goal is to only forward all your traffic, you can probably settle with just this proxy. External websites will see the IP address of your droplet or GCE instance or EC2 instance. This is extremely fast because the overhead is extremely small and it is very cross platform because everything supports an HTTP proxy.

Only, the only thing is, I had a nagging doubt about whether my HTTPS website headers were put in the HTTP Connect request that was made to the proxy server. I don’t know much beyond what I studied in my Networks course and what I learnt from the TLS RFC, but I had a feeling that the Host header was being passed through and that meant the HTTP request between the proxy server and my computer exposed the Host header (??). I wasn’t really sure about this. I didn’t ever set-up Wireshark and properly look at what was going on the network. All in all, this nagging in the back of my head ended up being the nail in the use-HTTP-proxy-for-everything coffin.

Enter, Wireguard

WireGuard is an extremely simple yet fast and modern VPN that utilizes state-of-the-art cryptography. It aims to be faster, simpler, leaner, and more useful than IPsec, while avoiding the massive headache. It intends to be considerably more performant than OpenVPN. WireGuard is designed as a general purpose VPN for running on embedded interfaces and super computers alike, fit for many different circumstances. Initially released for the Linux kernel, it is now cross-platform (Windows, macOS, BSD, iOS, Android) and widely deployable. It is currently under heavy development, but already it might be regarded as the most secure, easiest to use, and simplest VPN solution in the industry. – wireguard.com

This sounds simple, deceptively simple. I had tried to set-up Wireguard once before and I had failed miserably at it. I had run some commands on my personal computer which ended with the computer not being able to access the internet at all. I panicked for a short while as I found the opposite of every single command I ran to at least get internet back. After this experience, I had realized that Wireguard operated at a layer which below the application layer where things can get messy really fast and it’s hard to get out a problem you created with a command you don’t completely understand.

This time around, I had two advantages, I think. First, I had learnt about the value of taking notes and writing down every command I was running through the operations-type work I had done at work. And second, my personal computer was a Thinkpad with the latest Ubuntu and I was not worried about running into any hardware specific issues.

Step 1: Establishing the tunnel

Wireguard has a quickstart video. It is a side-by-side of two peers which starts from scratch and ends with a tunnel that connets the two computers. The two peers get their own IPs on the tunnel and they can ping each other. Now, I didn’t know how this would help me achieve my final goal of forwarding all my traffic through another peer, but it seemed like the first step in understanding wireguard and actually figuring out how to set it up properly.

I got a Digital Ocean droplet and I started running the two commands side-by-side. I was noting down the commands I was running and the output of each command as I went through the steps. I was also thinking about the resources I was adding to my personal computer and the commands that I would have to run to remove them. When I installed wireguard, I was asked to configure secure boot and restart the computer and enter the secure boot password. This was probably because wireguard is installed as a kernel module and secure boot doesn’t like additions to the kernel outside of the boot process? This is all complete conjecture.

Anyway, after restarting once to set-up secure boot and once so that the network interface device type wireguard becomes available to ip link. After this, I continued to run the commands just as they are run on the side-by-side video.

At the end of that process, I had a tunnel between my computer and the droplet. ping 10.0.0.1 started working as it should. At this point, I checked the output of ip route and realized that all the traffic to 10.0.0.0/8 was being sent through the wg0 network interface. I realized that I could do ssh 10.0.0.1 to SSH into the droplet! This was my first moment of epiphany; I suddenly realized that the droplet might well be next to me in my house. My computer and the droplet were on a network and knew about each other as 10.0.0.1 and 10.0.0.2. They could do things like move files across this network and it will all go through Wireguard’s crypto and will not be visible to anyone else on the network, not Digital Ocean, anyone in between or my home internet provider!

I hadn’t configured forwarding all my traffic yet, but I at least had a skeleton around which I could work. Also, I had learnt a lot more about ip link, ip addr and iptables, although I didn’t understand exactly how they all worked together with Wireguard.

Sidenote: I installed docker and squid-proxy on the droplet and forwarded all my traffic through 10.0.0.1:3128! This worked as I expected it to and changed my public IP.

Step 2: Watching the Wireguard talk

It is incredibly rare that you learn more about a piece of software after setting up one part of it. Wireguard is one of those rare pieces of software that claims to achieve something so simple that you have to actually listen to someone explaining it to believe it.

In this talk, Jason (@zx2c4) explains what Wireguard does and how it achieves the goal it was built for: establishing a tunnel between any two computers, as a kernel module. In particular, this diagram clarified a lot for me:

In particular:

I had read in a couple of places now that when adding the server as a peer on the client computer, setting allowed-ips = 0.0.0.0/0 will forward all of your computer’s traffic through to the server. Now, I realized that when the packet reaches the wireguard interface wg0, the packet’s destination IP will be matched with the allowed-ips list to find the peer. If I wanted all traffic to go to the same peer, I would simply add a single peer and set allowed-ips = everything.
On the way back, the packet would be headed to the client computer and the server configuration has the appropriate configuration already to route the packet properly. So, no changes were required on the server side.
“Linux kernel: Ordinary routing table -> wg0”: I had not done anything for this to be the case yet. I found this guide on the wireguard website which showed me how to set up my routes so that everything would go through to the wg0 interface. I chose to go with the second approach of adding two rules that forward all traffic to wg0 and then forward everything from there to the server.
The server gets all these packets on the wg0 interface which it has to then send outside through the eth0 interface. This is achieved using some iptables rules. I don’t fully understand these rules, they somehow work without specifying the name of the source and the destination interface.

Step 3: Forwarding traffic without name resolution

After watching the talk, I removed my existing peer and re-added the peer with allowed-ips = 0.0.0.0/0. Then, I added the appropriate ip routes using the NetNS guide on wireguard.com. The final piece in this puzzle was the changes on the server side to move traffic from wg0 to eth0.

After trying commands from several tutorials, I finally found the set of three rules that worked for me on a GitHub readme written by someone who was talking about a very specific use-case for wireguard!

iptables -A FORWARD -i %i -j ACCEPT
iptables -A FORWARD -o %i -j ACCEPT
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

Now, finally, curl <IP-for-icanhazip.com> returned the IP address of the droplet and not my local computer. There was no DNS server that my computer could use just yet, so there it was not able to convert the domain’s name into the IP address.

Step 4: DNS

Most of the wireguard setup guides recommend using Unbound DNS server installed on the VPN server. The configuration for Unbound looked long and I wanted to get something up and running before figuring more stuff out, so I decided to use Google’s DNS server. I got this command from the wg-quick utility that is packaged with wireguard-tools.

$ printf 'nameserver %s\n' "8.8.8.8" | resolvconf -a "tun.wg0" -m 0 -x

Sidenote: There are several other useful commands in the code for this utility. I have been reading it in my spare time to figure out what other things I could do to improve my setup.

After setting the DNS server up, curl icanhazip.com started working. Other websites also started working properly.

Evaluation

Checking speeds on fast.com showed me that using the VPN caused a slowdown of about 20 times - down from 21 MBPS to 1 MBPS. This is pretty good. Also, as literally all my traffic was going through wireguard now (except for DNS), I was able to use all other application without any special configuration. For eg, ssh started showing the droplet’s IP as the source IP in who -a, git push wanted to add a new origin because the push for my SSH key was coming from a new computer now. In general, everything seemed to work without anything breaking catastrophically.

Conclusion

At long last, I had a fast, working VPN solution for a Linux desktop computer. Wireguard is revolutionary. It creates a point-to-point tunnel between two computers (networked devices). What this tunnel is used for, is left to the user’s discretion. Wireguard simple creates the tunnel and gets out of the way. The only caveat is that one has to understand quite a bit about networking on Linux computers to use it properly. As the creator of Wireguard says in his talk above, “There’s no way to misconfigure wireguard and still have it work. If you don’t configure it right, it simply doesn’t work”

Next steps

For me, these are the things I want to do from here:

Install Unbound and start using it as a DNS server
Set-up Wireguard on my phone
Set-up wireguard on an OpenWRT router which moves the layer one step away from the computer and makes it easier for phones to connect and use wireguard
Read the Wireguard whitepaper

More about vim's formatlistpat, formatoptions and comments options

2019-11-02T00:00:00+00:00

I recently changed my vimrc file to set the formatlistpat variable explicitly for markdown files. Before making this change, I had to dive into the vim help pages and find out what option I really needed to change to get un-ordered lists to start working write.

The Problem

I like Markdown a lot. I take all my daily notes at work and at home in Markdown. It works well for me. Also, I recently discovered the Docker image that will help me convert from Markdown to any format effortlessly using a simple docker oneliner and the docker image jagregory/pandoc-docker.

One of the things that’s literally a part of everything that I start writing is a list. My over-all file is structured as a list, so I care a lot about the indentation of list entries. Before I made the change I started this post with, my lists looked something like this:

- This is a list item
- This is a very long list item that wraps
to a second line
- This is another list item

See how the second item starts from the beginning of the file and not from the first alphabet in the previous line? I realized that this was strange when I wrote this:

1. Task 1
2. Task 2: Keep track of the changes you made
   in Task 1
3. Task 3: Perform in parallel with Task 2;
   Might need help from someone else

and indented it with vim’s gq. I realized that Vim was capable of understanding lists and formatting list items properly. Every time I have tried to indent an un-ordered list in Vim with gq, it absolutely fails and I just have to go around and do it myself. But this great behaviour with numbered lists showed me that there some option I could change to fix this.

The Deep Dive

First, I started looking for an add-on that unordered lists well. I started with plasticboy/vim-markdown. It is one of the most popular markdown plugins for vim and I have never really tested it properly. So, I started off with it. But it didn’t really solve the problem, I got the feeling that it even made it worse somehow. I started looking online for other people who had run into this problem and this is where I started reading about Vim options.

I reached this incredibly enlightening vim-markdown issue: Rewrapping a bullet point inserts new bullet points => This was happening to me every time I tried to write a multi-line list item and wrap it with gq.

This issue had a few great vim options which were mentioned during the discussion around the bigger problem:

All three options seemed to be somehow connected to lists and their automatic formatting when writing the text, and their formatting when explicitly formatting with gq over that part or the whole file.

I started looking through vim options and trying to find where this difference between numbered and unordered lists stemmed from. The in-built :help command is really good and the descriptions of most of the configuration options are not confusing at all! Some of them are confusing, admittedly. But the descriptions for these three were at least fairly simple.

comments: A comma separated list of strings that can start a comment line

For markdown, the only character that behaves even remotely like a comment is the > character used to create a blockquote. Often when writing blockquotes which run multiple lines, it’s useful if the editor would just wrap the text and insert the > characters at the beginning of each line as blockquotes tend to be a single paragraph and a quick gqip fixing that is very useful. So, this was the first option I changed. I changed it to be just b:>. I don’t really undersatnd what the b: is, but this immediately fixed the formatting of numbered lists whenever I tried to do that! Unordered lists were still not working well.

Sidenote: Running :put=&comments puts the value of the comments configuration into the current buffer! I found this extremely useful when trying to inspect the configuration value of a regex without having to stare at the bottom line where the values are generally displayed.

formatlistpat: A pattern that is used to recognize a list header. This is used for the “n” flag in ‘formatoptions’. default: “^\s\d+[]:.)}\t ]\s”

Aha! formatlistpat and formatoptions are related! And the default regex is pretty easy to understand: it matches things of the form 1. or 1); i.e. numbered lists only! WHAT! I checked the value of formatlistpat for the markdown buffer I was writing my notes through this dive in:

^\s*\d+\.\s+|^[-*+]\s+|^[^\ze[^\]]+]:

I didn’t really understand this regex. I could see that it was trying to read numbered and unordered lists, but the third section in this regex with the \ze eluded me. In any case, I figured that if I could simplify this regex and write one myself which solved only the case that I cared about (unordered lists starting with a -), then I could change it later as I saw fit. So, I replaced it with just the middle part of the regex: ^\s*[-*+]\s. I inserted the unlimited number of optional whitespaces at the beginning of the line because nested lists are definitely part of my note-taking style.

Now, as I tried wrapping list items, I saw that it worked as I expected!

- This is a list item
- This is a very long list item that wraps
  to a second line
- This is another list item

I still didn’t quite understand the formatoption configuration though.

formatoptions: This is a sequence of letters which describes how automatic formatting is to be done. See |fo-table|

The formatoptions value in my editor was: tcqln. Looking at the fo-table, I saw that it did most of the things that I was taking for granted:

Auto-wrap at textwidth
Auto-wrap comments at textwidth but insert the comments character
Allow formatting with gq
Leave long lines as is when entering Insert mode
When formatting texts recognizes lists using formatlistpat regex

The meaning of the n option was particulary interesting to me. The help text clearly specifies that although the option is called n for “numbered list formatting”, lists are identified using formatlistpat which can identify any kind of list that you might come up with. (If you start your list items with letters, instead of hyphens or numbers, just set up formatlistpat!)

At this point, I had two more things that I wanted to do:

Improve the regex to recognize numbered lists
Improve the regex to recognize lists of TODO items in markdown ```sh
- Todo Item 1
- Todo Item 2: This is a long todo item ```

I am still working on the second part of this. For the point about numbered lists, I have really started re-evaluating if I even need numbered lists. I don’t ever write numbered lists if I can help it. (The ordering of lists is quite clear from the fact that we read text vertically downwards and it wouldn’t make sense for the third list entry to be the first point when writing the same list in an un-ordered format)

Another on-going problem that has been slighlty more annoying has been vim’s strange formatting whenever the content inside a parentheses goes beyond a single line or I end the previous line with a , (comma). I am not yet sure which option is causing this particular problem, but whatever that option is, I am going to find it and I am going to change it.

Managing DNS (or any other infrastructure) using Terraform

2019-10-13T00:00:00+00:00

Note: This post ended up being a lot longer than I expected it to be! It’s a post about why Terraform is great, even for setups where configurations don’t change often and how you can use it to share the access that you have with other people without giving them access to the actual infrastructure account. I will make a separate post about the specifics of importing DNS records from Cloudflare into Terraform and the process I used for that.

Terraform is great! It is a way to manage your infrastructure using a set of version-controlled text files with a plan command that tells you what is going to change and an apply command that applies the changes that the plan told you about.

Where terraform gets really good is when you store the Terraform State file in a GCS bucket and run the plan and apply commands in your repository’s CI.

It’s a loot of hoopla for DNS management of a personal domain. Is it worth it? Well, that depends on whether you have the time and motivation to set it up. Once you set it up though, it’s INCREDIBLY convenient.

What?

Terraform has three concepts that are pretty useful to know about:

Terraform state: This is the part of your infrastructure that has been “imported” into Terraform. Terraform manages the state of these “resources”. If you change any of these resoures from the web GUI and not through terraform, terraform has no idea about the changes you made and the plan will start getting weird.
Terraform configuration files: These are the files with the actual resources. Each resource has a type and an identifier inside terraform. And it has some arguments and attributes. Arguments are configurable and can be changed through these files. Attributes are read-only and generally include things like a generated ID which is used by the infrastructure provider to keep track of your resource.
Terraform providers: Providers are plugin-type software packages which form the interface between Terraform and the infrastructure provider. As far as I understand, Terraform understands the 3 changes that can be done on any given resource: create, update and delete.
- During plan
  - Terraform calls the provider and gives it the changes that were made to the terraform configuration files
  - The provider then calls the API of the infrastructure provider to find out what changed
  - The provider gives terraform the changes
- During apply
  - Terraform calls the provider and tells it the resources to create, update and delete
  - The provider calls the API of the infrastructure provider and makes the appropriate POST, PUT and DELETE/POST requests
  - The provider returns to terraform the actual changes made
Terraform backend: A backend is the location where the state file is stored. By default, this is local. You store and manage the state file yourself. You can also set this up to be in a cloud storage option like Digital Ocean Spaces or Google Cloud Storage. Storing the state file in the cloud allows you to give others access to run plan and apply commands.

So, in this manner, terraform has completed abstracted out the actual infrastructure being managed from the provider that it is provided by. Everything is a resource in the terraform world, and there can be only three operations to a resource: create, update or delete.

How useful is it?

Now, just having a log of all the changes you have made to your DNS records is pretty useful. You can go back and forth in time, you will never really have to deal with IPs again because of how easy it would be to create a new record, and so on.

The other features that terraform brings to the table are:

An audit log that can be seen by anyone with access to the repository
Collaborators who can add records without having access to the actual infrastructure provider account

Both the first and second are extremely useful for DNS management. I use Cloudfare as the DNS provider for my personal domain, and cloudfare has this annoying way of changing DNS credentials from one account to another account. You have to export the DNS records, import it into the new account and then change the NS records at the domain provider.

I tried to do that once for metakgp.org (another domain that my account is connected to) and it did not go well. Something went wrong, the transfer didn’t work and the website was down for about 12 hours. This wasn’t very catastrophic, it’s a college project and it went down at a time when very few people were trying to access the domain. Nevertheless, the whole point of that was to ensure that the DNS was being managed by someone who was in the college instead of someone who had graduated.

The thing about colleges is that people keep graduating! And few people in their freshman or sophomore year have any clue about what DNS is or what records are or how to change them safely etc. I learnt most of the things I know about DNS in my fourth year at college. So, this would be a perennial problem.

So, the basic thing that I was looking for from a move to terraform was a way to safely share access to the infrastructure provider with other people.

A completely local setup

Now, you can have a bunch of terraform configuration files in a repository alongwith the state file. You can run appropriate terraform plan and terraform apply commands manually. The state file is a file that has to be maintained separately from the configuration files.

One way to do it that’s fairly manual and works for use-cases where changes are not very common and are centralized with a few people is to keep the state file checked into Git. The state file is a JSON file and can be easily stored and versioned. The local flow would look something like this:

You import your DNS records into the local configuration files + the state files
You put this repository out there
People make a pull request with the changes they want to make
- You run terraform plan manually and post the output on the PR
You (and others) review their PR; you merge it when they look okay
Now, you (or anyone who has access) manually run terraform apply and apply the changes that were requested

This is already a great improvement on the nightmarish process of moving DNS management from account to account just to let others take over access. You have to remain involved in the process, yes, but at least others can review and all you have to do is run the plan and apply commands.

What if you use Continuous Integration?

Continuous Integration (CI) makes this whole process incredibly easy. It de-centralizes DNS management without de-centralizing control over the provider account. Here’s the flow with CI:

You import your DNS records into a terraform state and into local configuration files
You set-up Google Cloud Storage (GCS) to act as the backend for your terraform state
- Now, your state file is stored on GCS instead of being stored locally on your computer!
- Anyone with the right credentials can run the plan and apply commands now!
You set-up a CI pipeline where opening a pull-request runs the plan command and merging to master runs the apply command
- You have to give the CI pipeline access to the provider and the GCS bucket using a service account
- Both these things are fairly easy to accomplish
Now, you let others make Pull Requests
- When someone makes a pull-request, they can go to the CI pipeline and see the plan
- If the plan looks okay, they can request review
- Others review the PR and approve it if everything looks good
- Anyone with permissions to merge to master can merge the PR and have the changes show up in their infrastructure right away!
  - Note that the person who is merging doesn’t need to have access to the GCS bucket OR to the infrastructure provider! All they need to have access to is the repository that is maintaining the configuration.

This is the holy grail. A single repository on which users can be added and removed as required.

If changes you didn’t want to merge have been merged to master, you can revert the PR and merge to master again. VOILA!
If something goes wrong, you can lock down access
If you fear that the CI keys have been taken by a bad actor, you can remove that actor and roll them

Conclusion

Terraform lets you share the ability to make changes to your infrastructure with a wide group of people without giving them any direct access to the infrastructure itself.
Terraform lets you maintain an audit log of the changes that happened to your infrastructure
Terraform lets you roll-back unwanted changes with a simple revert PR

Compiling vim with python3 support!

2019-08-31T00:00:00+00:00

TL;DR

sudo apt-get install -y python3-distutils python3-dev
git clone https://github.com/vim/vim.git
cd vim
./configure --prefix=/usr/local \
     --enable-python3interp \
     --with-python3-config-dir=/usr/lib/python3.6/config-*
make
sudo make install

The Longer Version

Note: I put the commands at the top of the post because I hate it when people start with the story and the TL;DR comes at the end of the blog post. I strongly believe that the TL;DR of all blog posts and long text-filled posts should be at the top of the piece.

I started compiling vim from source a few months ago. See, I needed to get an extension called vim-search-me. This extension was calling a function called trim() but only newer versions of vim had this function. I opened a PR on the original repository to ensure that the call wasn’t made if the function didn’t exist, but I also wanted a version of vim which actually had this function so I could use the extension.

Digging into the vim issues, pull requests and source code a little bit, I realized that the function was introduced in a recent commit. I wanted to build a version of vim which was newer or equal to this and would allow me to actually use this function.

Surprisingly, the first time around, building vim was extremely easy. I downloaded the source code using git clone, switched to the appropriate release tag, configured the build with a prefix of /usr/local (./configure --prefix=/usr/local), make and sudo make install! Voila, I had a custom built version of vim! The part I probably “dig”-ed the most was that the vim --version output actualy printed the name of the computer and the date/time when I compiled it! This was pretty interesting and I realized that now I could get patches that were made on the original vim code base whenever I wanted. There was no need to depend on the vim that comes packaged inside the apt-get repositories or inside macOS.

Requiring Python support

At work, I write a lot of Go code and almost all the code has to have tests. The code coverage threshold is around 80%. So, adding any new function immediately necessitates the addition of a test for that function. Now, there’s a way that tests are written for Go functions and I was beginning to realize which pattern I liked and it turns out I was re-writing a lot of the templating code to set-up the test, every single time I started writing a new test.

UltiSnips is a vim extension that lets you store Language specific snippets which can be inserted into a file when the filetype matches the snippet you have stored. I heard good things about the extension from my co-workers and found it in some of their dotfiles repositories! I installed the extesnion and tried to start vim and it said that it needed python support! I hadn’t compiled vim with python support, so I had to find out how!

$ vim --version | ag "python"
+cmdline_hist      +langmap           -python            +visual
+cmdline_info      +libcall           -python3           +visualextra

Finding the correct `./configure` flags

The configuration script output prints all the flags, their values, and the output when the checks required to enable their inclusion in the build were performed. The python related flags were --enable-pythoninterp and --enable-python3interp. System python on both my work computer and my home computer was python 3.6, but I didn’t want to take a chance. So, I explicitly specified python3 and ran the configuration script.

On my work computer (a Macbook Pro from last year), supplying just the --enable-python3interp to my configuration script made everything work fine. I didn’t actually realize that something else might be necessary because it worked fine. make and sudo make install and I had Python support in vim!

On my home computer (a Thinkpad) with Ubuntu 18.04, things were slightly more involved.

The first error I saw with just the --enable-python3interp flag was the configuration script’s inability to import distutils.sysconfig inside Python3. A couple StackOverflow answers later I realized that the python3-distutils package installed that python3 module.

$ dpkg -l | ag "python3-distutils"
$ sudo apt-get install -y python3-distutils
$ python -c "import distutils.sysconfig"

This error being fixed now, I ran into my second error. This was a slightly weirder one. It said that it couldn’t find the configuration directory for my Python installation. This StackOverflow answer pointed me to a configuration script flag which allowed me to explicitly specify the path to my Python configuration directory.

The third error I ran into was stranger:

Unable to find the Python.h file inside the python configuration directory

And it was showing an #include <Python.h> line from some file containing C code. The above answer also had a line about having to install the python2-dev package using apt-get because it isn’t pre-packaged inside the Ubuntu installation.

On Ubuntu 10.04 you have to install ‘python2.6-dev’

So, I installed the python3-dev package.

$ dpkg -l | ag "python3-dev"
$ sudo apt-get install -y python3-dev

After this, the configure command worked and succeeded! The python related lines were succinct, and didn’t really indicate success. But they didn’t indicate failure either. So, I ran make. make creates the binary inside the vim/src folder. Running the binary from there will immediately tell you if the feature is installed or not.

$ ./src/vim --version | ag "python"
+cmdline_hist      +langmap           -python            +visual
+cmdline_info      +libcall           +python3           +visualextra

I had built vim with python3 support! Now, the lines at the top of vim --version which tell you when vim was last built and who built it:

$ vim --version | head -n4
VIM - Vi IMproved 8.1 (2018 May 18, compiled Aug 31 2019 11:32:55)
Included patches: 1-1949
Compiled by siddharth@home-thinkpad
Huge version without GUI.  Features included (+) or not (-):

Firefox Add-on to jump from GitHub to CircleCI extremely fast!

2019-08-25T00:00:00+00:00

What?

I use GitHub and CircleCI at work. One of the most common workflows for me, is to merge a PR, click on the repository name to go to the repo page, scroll down to the Readme and click on the CircleCI indicator to check the status of the CI job kicked by the merge.

When the CI workflow is done and completes successfully, I go back to the repo page and click on “Releases”, create a new release (a Git tag) and then again go to the readme to go back and wait for the Release CI to finish.

Once that is done, I open Spinnaker and wait some more for that to get triggered. For some reason, Spinnaker takes about 10 minutes to detect that a new image with a tag matching a provided regex was pushed to GCR.

In any case, you might have noticed that I switch between GitHub and CircleCI a lot. Now, most repositories at work have a CircleCI indicator at the top. If they don’t, there’s always the option to open a PR and wait for someone to approve it and then you can finally merge it. But this whole process was extremely annoying to me, because there’s no need for this indicator.

You see, the link patterns are fixed and extremely easy to generate using just regex!

GitHub: https://github.com/username/repository/.* CircleCI: https://circleci.com/gh/username/repository/

So, all you really need to do is replace the beginning and remove anything after the end of the repository name: that’s a textbook usecase for a bookmarklet with JavaScript!

Why not a bookmarklet?

So, I wrote a bookmarklet. Very short and sweet!

javascript:
var url = location.href;
if (url.match(/https:\/\/github.com\//)) {
    var baseRemoved = url.replace('https://github.com/', '');
    var components = baseRemoved.split('/');
    if (components.length >= 2) {
        var account = components[0];
        var repo = components[1];
        var newURL = "https://circleci.com/gh/" + account + "/" + repo;
        location.href = newURL;
    }
}

I add it into a bookmark and click it. Nothing happens. The console shows me an error that sort-of tells me something useful, but not really:

CSP shouldn’t affect bookmarklets, but it does on Firefox, for the time-being at least. The issue in Firefox that stops the bookmarklet from running has been going on for nearly 6 years now! That’s an awful lot of time people have been dealing with this.

The Add-On!

Extensions have more permissions than a bookmarklet. A user consciously installs them and they are reviewed before they go on the Firefox add-on store. I started looking through the add-on developer documentations to find the right APIs that I wanted. Originally, I was looking for a keyboard shortcut. But I realized that that would be tough and that I should stick to doing a clickable element before aiming higher.

My next plan was to add a fixed HTML element on all GitHub repository pages which would give me a reliable method to switch from GitHub to CircleCI no matter which computer I was on. This reliability was particularly important to me, I wanted the element to be in the same place on all GitHub repository pages.

Turns out the way to do that is something called a page action! It looked perfect for my use-case. It was a button near the address bar. As a cherry on top, I could connect a keyboard shortcut to it using a command construct!

The Firefox add-on documentation is extremely good! They have example extensions which use each JavaScript API that is supported, and for pageAction it was apply-css. Going through this extension’s background.js and manifest.json, I was able to quickly come up with a dev version.

After that, I cleaned the code up some more, made sure that it had icons and the other images that were required were present (I used Gimp for a few minutes. By any designer’s standards, this icon is probably hideous)

After pushing the add-on code to GitHub, I created the add-on bundle and uploaded it to the Firefox store. I did that at 2019-08-24 17:00 JST (UTC+0900). It’s been 24 hours, and I have gone from spot 22/22 to 5/22 now. I don’t know if the review is taking longer because it’s the weekend or what.

Bonus: A Chrome extension! What?!

Most of my co-workers use Chrome and I have read a little bit about the WebExtensions API and how it’s a standard and can be used across browsers and what not. So, I wanted to give it a quick try and see if I could make it work on Chrome with a few tweaks. It turned out to be fairly easy! The Chrome store forces on you several severe restrictions:

A 128x128 icon which has 16 transparaent pixels on both sides
- Why must Google force the developer? Surely, they can pad the image once it has been uploaded and transform it to whatever size they want as long as the aspect ratio is 1:1 (Square)
A 1280x800 screenshot
- Funnily enough, they will NOT accept ANY size except 1280x800. That I find very hard to digest because most HD screen-shots are 1920x1020 which can be naturally resized to 1280x720. This 1280x800 thing is particularly strange.
- Anyway, I opened up Gimp and used the Align tool to make an image that size.
- P.S. I am really starting to warm up to how useful Gimp has been lately!

Once, I jumped through those hoops and had the right images, I was able to submit my add-on and voila, there my extension was! No review process to get things on the Chrome Extensions store. That would make me squeamish, if I was using Chrome as my daily-driver and had a couple shady extensions.

Edit 2019-09-15

Firefox 69 was released a few days ago! Now, bookmarklets to change the URL of a website work!

Setting up Ubuntu 18.04 LTS on a Thinkpad

2019-07-27T00:00:00+00:00

It’s been a while since I have setup a brand new computer. I did set up a Mac when I started working in September last year, but that was more of a step-by-step thing where I just started with my existing dotfiles and picked up stuff along the way from people around me who had been working for a much longer time and knew of so many great Vim extensions and tools that I had previously never heard of. Prezto was one of them!

I was going to install Ubuntu 18.04 LTS on an IBM Thinkpad L390. It was released around December last year (or earlier this year?) and sounded like a really good handy laptop for travelling, particularly. I bought it with a mid-range RAM config of 16GB and a 512 GB SSD. For comparison, my current personal machine was a Dell Inspiron from 2015 with a 1TB HDD and 8 GB of RAM; so this was a pretty significant upgrade.

TL; DR Installing Ubuntu took about 5 minutes flat. I am not even kidding; I never realized that copying files and cleaning up the disk is one of the slowest steps in an install and is a complete bottleneck! Also, the partition setup was much simpler for me this time. 80 GB Windows 10, 80 GB root / and the rest for /home. That’s it. I decided to keep it extremely simple.

Deciding to keep Windows 10 was a pretty big decision, and I definitely took the safer option. I was particularly worried about warranty issues if I nuked Windows because I am convinced that any service center would only have trouble-shooting software for Windows. In fact, every time I have taken the Dell in for a repair, I always boot them into Windows and leave them in there. So, even though I was looking forward to nuking Windows and having a pure Ubuntu single-boot machine, I decided to continue with Dual Boot. My concern of Windows being at the top of the Grub screen had gone away since my last install of Ubuntu and subsequent discovery that you can actually control the order in which the operating systems show up on the Grub screen!

Once the installation was done, I set about getting all the software that I considered essential. The last time I did this with the Macbook, I didn’t keep track of what commands I was running and this was a mistake. So, this time, I took copius notes.

I also added this gist to my dotfiles, just in case I ever have to setup an Ubuntu computer from scratch again. (One of the steps I was fumbling at was linking the .z* inside $ZDOTDIR. I hadn’t linked these files and didn’t realize that prezto won’t work without them!)

I have been using this Thinkpad for about 2 weeks now, and everything has been going well so far. WiFi, sound, the touch pad, brightness and volume controls everything seem to be working fine. I haven’t run into severe Ubuntu-Thinkpad compatibility issues yet. And hopefully, I won’t run into them far into the future!

Ongoing issue with Firefox add-ons due to an expired certificate

2019-05-04T00:00:00+00:00

Note: This is an ongoing issue at the time of writing (19:30 2019-05-04 JST). The team at Mozilla is aware of this issue and is working hard to fix it as soon as they can. You can track the bug here and on the IRC channel #firefox on irc.mozilla.org.

Detection

The issue started around 6 pm PST on the 4th of May when some people noticed that their add-ons were disabled by the browser, though the add-ons were working just fine just a few hours ago. (Some people had theirs stop working during on ongoing session!) This led to the opening of a GitHub issue and a bug on Mozilla’s bug tracker for Firefox, Bugzilla.

I ran into this as soon as I woke up this morning and opened my work computer to start my on-call shift. I didn’t know I had a pretty interesting on-call session in front of me. On a 12-hour on-call shift, I didn’t have much to do anyway. So, I ran through what the issue was, how it started happening, and what people are saying about it.

Sidenote: I set up IRSSI because it felt like a lot of the discussion was happening on IRC and not on GitHub or bugzilla (mainly because the developers inside Mozilla locked the GitHub issues and restricted who could comment to keep those channels clear; as more and more people started to figure out that what they were running into didn’t have an easy fix, these pages would only have started getting more polluted and the main problem would have become harder to find)

IRC was filled with a bunch of people who were talking about where the issue started and why it was allowed to happen. One of the most common threads of conversation was: “This feels like we are using a product built inside a walled garden”. Firefox devs continued to tell the world that they knew about the problem and were working hard to fix it but what this fix was and how they were approaching it was not clear to anyone on IRC.

As for the issue itself, one look at the title clarified what it was:

All extensions disabled due to expiration of intermediate signing cert

Let’s delve into what exactly this means!

Mozilla Add-on signing

Mozilla has a great wiki on how they sign add-ons. The add-on submission process goes like this:

Developer submits a plaintext XPI to Mozilla Addons
Mozilla analyzes this XPI for security vulnerabilities
When approved, Mozilla signs this package
1. Mozilla has a root CA certificate that’s shipped with the browser (R)
2. Mozilla has an intermediate certificate that’s used for add-on signing (I)
3. Mozilla derives a new certificate from this intermediate certificate - the end-point entity certificate (E)
4. Mozilla signs a file with cert E
5. The package is shipped with this signature (from cert E), the cert E, and the cert I.
After downloading an add-on, the Browser verifies this signature:
1. Verifies that the signature was made by cert E
2. Verifies that the cert E has a valid chain back to the root cert
3. Verifies that the files match what was signed (using a bunch of hashes)

Our current problem: The intermediate certificate, cert I, has expired!

Now, the certificate itself is available in every single add-on, and can be inspected. So, that’s exactly what I did.

The Certificates

To get a certificate, you must get an add-on package; I chose uBlock because that was the first thing I noticed when I was looking for one.

Get the XPI for any add-on
Unzip this xpi file using unzip -d extension
The signature is inside the file META-INF/mozilla.rsa
Use openssl to inspect the certificates!

openssl pkcs7 -in META-INF/mozilla.rsa -inform DER -print_certs -text

You can find the output of this for the uBlock extension here.

These lines inside the intermediate certificate (second certificate in the certificate chain are the culprit of the present issue):

Data:
    Version: 3 (0x2)
    Serial Number: 1048580 (0x100004)
Signature Algorithm: sha384WithRSAEncryption
    Issuer: C=US, O=Mozilla Corporation, OU=Mozilla AMO Production Signing Service, CN=root-ca-production-amo
    Validity
        Not Before: May  4 00:09:46 2017 GMT
        Not After : May  4 00:09:46 2019 GMT
    Subject: C=US, O=Mozilla Corporation, OU=Mozilla AMO Production Signing Service, CN=signingca1.addons.mozilla.org/emailAddress=foxsec@mozilla.com

The validity of the intermediate certificate ended today. The actual expiry was about 1 hour ago, but this issue has been on going for nearly 9 hours now, so I don’t know what that is about. Hopefully, we get a post mortem from Mozilla once the issue is fixed and we can all find out exactly what the issue was!

The Solution

I haven’t worked with CA and certificate systems for quite a while now. But as far as I can tell, this is the solution that comes to mind:

Issue a new intermediate certificate
Re-generate and re-sign all add-on packages
Upload these new add-on packages with the signature to AMO
Get everyone to download the extensions again OR release a special patch to the browser which downloads the add-ons again for you.

Interestingly, Mozilla was able to release a work-around for people who didn’t have their add-ons disabled. They said they did that here. One of these users was on IRC and they shared their certificates for the uMatrix extension here. These still expire at the same time. So, Mozilla was able to find some sort of quick workaround which helped them make browsers ignore the results of the add-on verification test.

Edit: I was enlightened by a user on IRC that the fix they deployed was a Study. These can be seen inside about:studies and this particular hotfix is called hotfix-reset-xpi-verification-timestamp-1548973! Nifty!

P.S. I wrote a post a few months ago about how cool openssl is! I was finally able to use it in the wild with this issue! Check out the post here.

Oh-My-Zsh to Prezto - 4 seconds faster

2019-04-06T00:00:00+00:00

I have heard a lot about prezto. People who work with me use it and we got into this recent discussion about how slow a curl request inside your zshrc makes your shell. Strangely enough, I hadn’t noticed that it made the shell slow at all. In fact, I was saying that it didn’t make it any slower than without it. I removed it from my zshrc and checked it and indeed, it didn’t make my shell any slower.

A few days went by; I was thinking about this when I was next thinking about why I hadn’t noticed this clear increase in the time. So, I looked at the time my shell took to start up again. It was … interesting: 6 seconds.

YUP. 6 seconds on a machine with an SSD, 16 GB of RAM and an i7 processor. So, what is the problem here?

The Search

I have a zshrc that sources several other files when it loads. Going through my main zshrc and all those zsh files, I saw that everything in there was a definition: export, alias, function. Apart from this, I was sourcing a few other things: fzf and oh-my-zsh. Testing these separately, I could see that these were extremely fast and not the culprit.

That’s when I realized that I would have to profile my zshrc

echo "START: `date +%s`"

st=`date +%s`
# op
endt=`date +%s`
echo "op_name: $(($endt-$st)); $st -> $endt"

I added a bunch of blocks like the one above to my zshrc. I sourced a few times. I kept adding and removing these blocks. I eliminated the sourcing of the 18 or so zsh files in my dotfiles -> They were all happening within the same second. I looked at the plugin loader, the definition and sourcing of some add-ons like travis and cargo, but they all happened very fast too.

The strange thing was: I couldn’t find the culprit. Every step that I profiled happened very fast, and YET, the whole process took nearly 6 seconds. What was going on here?

The Realization

I started searching for reasons and people running into this hurdle before. I had prezto in the back of my head, so I searched for that and comparisons too. That was when I reached this amazing article. The author put it quite succinctly:

Zsh is an amazing shell, … BUT. It’s God. Damn. Slow!

Those were my feelings too. I have used oh-my-zsh for nearly 5 years now, it has served me very well. It has helped me get through some tough times (I am almost certainly romanticising); But it was time.

I read another article. The steps were very simple. My configuration would work as is. The one thing that wouldn’t work would be the prompt. I would have to find a prompt I liked; Or, if it was un-obscure enough, recreate my beloved ys on prezto. (More about why I love this prompt so much in the Postscript)

The Switch

Having mulled over this enough, I decided to go for it. It was quite clear in my head, the only things that could really break were my prompt and my color scheme. Both things that couldbe reverted by cleaning everything up and making a new install of oh-my-zsh and trying another weekend. With that re-assurance, I copied my existing ~/.oh-my-zsh into a backup folder, commited and pushed everything in my dotfiles and began.

git checkout -b switch-to-prezto

The process was fairly simple. After uninstall_oh_my_zsh and installing prezto using the commands on the readme, I had a running shell. My zshrc was working well. Perhaps the thing that most surprised me was that the color scheme was unaffected! I have dealt with fixing the color scheme before and it was incredibly obscure, undocumented and painful. So, that is one thing I was extremely grateful for.

Having ensured that the shell itself worked without any of the stuff on top of it, I started looking at the list of plugins (prezto calls them modules) that I wanted. I ensured that I was getting fish-like auto-suggestions before I even contemplated this move because that is one of the most useful things EVER! (It’s so incredibly fast; compared to even fzf)

I included the ones that I absolutely needed and auto-suggestions and syntax-highlighting started working! It was surprisingly easy and in fact, easier than oh-my-zsh which is so popular because it is easy and versatile. I had quite a bit of trouble setting up the right colors in oh-my-zsh with auto-suggestions because the color has to be dimmer than the color you are typing in but at the same time, it has to have enough contrast with the terminal background to ensure that you can see the suggestion without straining your eyes. I had hit the right balance with this with a slight change on top of solarized-dark.256 and I didn’t want to go through setting this up again. Thankfully, prezto worked fine the first time!

At this point, I had a working shell with all my aliases and functions; vim mode in the shell was working fine too. The experience till now had been good. The one thing left to configure: The Prompt.

The Prompt

TL; DR I couldn’t find the prezto equivalent for ys, so I wrote a prompt that looks exactly like ys.

I like ys because it doesn’t have any crazy characters. The green “o” looks just as gratifying as the green tick mark. The red “x” is almost indistinguishable from the red UTF8 “x” mark. One of the main reasons I chose ys was because it has only ASCII characters. That ensures that it looks good everywhere: Ubuntu, Mac OS, even Windows would have worked fine (if I had ever tried to make it work there). There was no ys clone for prezto, which was quite surprising; I was confident I would be able to get away copying some text around without having to understand how prompts are written. Hard luck there!

So, I started writing this prompt. I copied ys’s code over to a file which I thought would make it work. It was actually surprisingly close to what I wanted. The colors were off because it seems there’s a slight difference in how colors are formatted in oh-my-zsh prompts and prezto prompts:

oh-my-zsh: ${$bg[white]on git:%}
prezto: ${%F{white}on git}
oh-my-zsh: %{$reset_color%}
prezto: %f

After changing this around, things started looking very similar! In fact, the only thing left was the git-related info in the prompt.

At this point, I realized that I would have to go deeper into the prompt world or figure out the correct piece of code to copy from an existing prompt module. I couldn’t make the latter option work, so I decided to delve deeper into the prompt world.

Reading all of those READMEs and looking at some of the existing prompt files, I figured out how the flow of prompt files works. And ended up creating the exact same thing that I had on oh-my-zsh! And I’m mighty proud of that!

The Result

At long last, I had reached the point where my prompt looked and worked as I wanted it to. It was time for the test:

$ date +%s; source ~/.zshrc; date +%s

For some reason, the above command wouldn’t print the time after loading the shell. So, I measured the time to start a shell by opening a tmux pane or an iTerm2 tab: 2 seconds!

That’s 4 seconds saved every time I decide to open a new pane in tmux!

My fork of prezto is here. The prezto equivalent of ys is here.

This is a completely new shell and I am surprised at how similar it feels to oh-my-zsh and I don’t feel like I have to make any more changes, already. I might make some minor changes though, because using the git module brings in a bunch of git-related aliases that I don’t particularly like having around. (Eg: gcR removes the top commit from your branch. It’s a soft reset and is probably useful, so I might keep that one around. But there are a bunch of others listed here that I don’t like having around)

Postscript

This has been the third project that I have done something real on which will not get merged into the main project but will only be used by me. I am sort of having a re-realization about how powerful open-source software is. These are all fairly small projects. Being able to make these small changes in them to make your workflow a lot faster and also give you the satisfaction of understanding the tools you use on a daily basis is pretty empowering. All you really need to be able to do this kind of thing is a some know-how about the language that the tool is written in, the time to do it and the motivation to.

Doing this on a small scale, also kind of makes me wonder about the power of being able to do this at a much larger scale: an embedded software engineer’s ability to make changes in the http library or a Linux kernel developer’s ability to tweak things to be just so.

True Detective Season 1

2019-03-10T00:00:00+00:00

This post does not have spoilers about True Detective Season 1

When Season 3 started recently, I found myself thinking about what exactly it was about True Detective season 1 that I liked so much. This post is a look into what’s great about that season.

The Plot

A murder mystery in Louisiana that is being investigated by two very different detectives: Rust and Marty. Marty is every man’s man; he has a family, two daughters and some mistakes in his character. He mingles with people well and has a good rapport with the people he works with.

Rust is single, he has a mysterious past and no one really knows what he did before coming here. He waxes philosophical almost all the time. Most of the people that Rust works with don’t really like him and are unsettled by him. Early on in the series, we are shown how Rust has insomnia and works through the night a lot of days. He is also an excellent detective with an uncanny ability to look through people and situations. One of his most iconic quotes: “I’ve never been in a room more than 5 minutes and not known whether they did it or not. How long does it take you?”

The original murder mystery happens in 1995. The two men are being interviewed in the current time (~2012) about this mystery because something similar has apparently happened. The series unwinds the complete story of the murder mystery, how the two men went about investigating that back in ‘95 and it’s connection to the mystery at hand.

That’s the story. There’s nothing particularly remarkable about the story when you look at it. The way it is shot and told is extremely well-done. Although, the story sometimes alternates between the present, the past (‘95) and the more recent past (2002), I have never felt disoriented about what is going on. Every single tool that is useful in telling the viewer about the different times is used effectively and constantly: the lighting, clothes, the characters’ ages, the characters’ traits and how they change over time. That’s one of the reasons I like this show: Despite having a complex plot, the viewer is never disoriented or disconnected from the characters. We know exactly what is going on, all the time

Next: Rustin Cohle. Rust is the central character in the series. In some ways, the murder might be the central theme of the show, but Rust is the driver. He decides where the show goes, he decides the mood of the show. His quotes are legendary;

If you have ever seen an impressive character on screen, you know that they are always charismatic, strong, and have powers that other characters on screen don’t. Rust has all of this. He is stronger than Marty, he can read people and never really has to think too much about it. All of these things make him less and less relatable; he’s depicted to possess super-human abilities. But there’s a raw human quality to him: From the beginning, his insomnia, his addiction, his inability to engage socially, some of his traits clearly show how lonely and tired he is. He talks a lot about existence and it’s usefulness and how “consciousness was a mis-step in evolution”; you wonder if he’s talking about all this because he can’t find the point of his own existence. Marty recounts this when he says to him, “When you talk like that, you sound panicked”.

Rust is one of the most well-written, well-directed and well-acted characters in the “investigator” demographic. Another detective I like is Carrie Mathison from Homeland. She has a lot of the same qualities; minus the existential talk. And, while Carrie is dealing with stakes that are as high as saving thousands of people, Rust is working on stopping someone who is hurting a few people. This smaller focus makes him more human.

The final dimension to Rust’s character comes from the way people in the show look at him. They are in the same world and were purposefully written to react to him the way they do. That gives us the unique perspective of what the writers wanted Rust to be seen as. Here, one of Maggie’s quotes is very helpful, “Rust knew exactly who he was and there was no talking him out of it”. She finds Rust to be someone who knows who he is, who is done figuring out the purpose of his existence. What’s remarkable is that, Marty, who has spent a lot more time with Rust doesn’t actually believe that. A superficial look at Rust might lead us to believe that he has it all figured out (like Maggie believes), but we know that when Rust talks about himself and from his actions, he is far from that point of stability and security. There is a duality about him: superficially, he is a super-hero: he is strong, intelligent, good at his job. Get to know him a little better, and you realize that he has existential thoughts, his life revolves around his job, he has trouble figuring out his social purpose. “We are the bad men who keep other bad men from coming to the door”

Podcast Review - January 2019

2019-02-06T00:00:00+00:00

Happy New Year! Let’s jump right in!

This American Life - The Walls - 5/5

A fascinating episode! This American Life is one of those podcasts that I listen to rarely, but every time I do I end up with an episode that is absolute gold! This episode came out at the top of the “Trump shut down the government for a wall” news story and debates some of the amazing walls around the world. The conclusion is simple: Walls make us feel safe, irrespective of how big they are, whether there’s another wall on the other side of the wall on our side, whether it actually even exists or has been conjured out of nowhere just to pacify us. This episode also follows the story of a man who has lived all his life with a wall in his home country; A wall that has made his relatives strangers to him and how he reconciles with that. Once a wall is built, it becomes a fact of life and affects the way life is lived around it When put like that, we really see how much power walls have! An absolutely amazing episode!

The Slow Burn - Season 1: Watergate

The Watergate scandal is something that was shrouded in a lot of mystery for me before I heard the first season of this podcast. The scandal is mentioned so often no matter which country’s news channel you turn on. It’s this daring attempt at covering up something that so many people knew that anyone who must have looked at it from the outside, in the aftermath, would have said: “AH! That could never have been kept under wraps” and yet the truth is, that it was kept under wraps for quite a long time. Listening to this podcast really makes you think about what would be discussed as history 50 years from now when other people are analyzing the choices that were made by us. A rivetting story that tries to get to the scandal from as many angles as possible!
Modern Love - What The Sea Took Away

This episode talks about how to deal with loss, loss that is inexplicable. Loss that doesn’t come with the elusive concept of eventual closure.
Modern Love - A Young Man’s Game

An episode about how bearing children is a young man’s game. It’s funny and evocative. One of the quotes that stayed with me after I finished listening to the episode: “I traveled and I wrote books, and I loved that life. But I truly regret that Larkin hasn’t gotten to know to know my mother, other than through our stories”
Modern Love - How To Break Up With a 2 Year-old

Another amazing Modern Love episode! I have really been sucked into this podcast, it’s amazing essays and narations! This episode is about how babies attach so easily and so effortlessly with adults and even as they grow up to forget these adults and how important they were to them as babies, the adults keep getting older and realizing more and more how insignificant a part they were of this young human being’s life! Heart warming and heart breaking, in just a few paragraphs. If you can’t listen to the episode, definitely read the essay.

This is end of post 4 in this series!

Podcast Review - December 2018

2019-02-06T00:00:00+00:00

Note: This post is very delayed (it should have been out on December 27th); It was stored as a note on my iPhone with a set of links. I have added a short summary of each podcast and why I liked like it other posts in this series. Let’s dive in!

The Daily - Ethics of Genetically Editing Babies

This is related to the story where a Chinese scientist claimed to have created / helped make (what even is the right word?) the world’s first genetically edited babies. After discussing this case and talking about what actually happened and why it might be suspect, the guest on this show goes on to describe why one or two reckless scientists like this person can actually end up forcing countries around the world to heavily regulate or even completely stop the research into this field because they don’t trust the scientists anymore. I think that that was a very succinct explanation of why things like this are bad, while also keeping in mind that more and more of the things that we consider impossible in our lifetimes will become possible and someone will do them. And our tolerance of innovation might very well be defined by our acceptance or rejection of these ground breaking events.

The Daily - The Business of Selling Your Location

Nothing new here for anyone who has given thought to how conglomerates like Google actually make money (from services like Gmail, Search, Calendar and Youtube) which are used for free by most people. This episode does mention some companies that collect a lot more than is clearly stated on the app (no surprise there) and how that data, if looked at person by person, can create a pretty good map of their life: where they live, the restaurants they frequent, the people they meet up with, and so on. P.S.: Stop using Google and all of it’s products; Pay for e-mail, use an open source browser. Those are some of the steps that I have taken to feel better about the providers I am using day-in-day-out.
Future Perfect - How to be a better carnivore

This is a great episode about how eating meat or fish isn’t the worst thing in the world. Lately, I feel like there has been this (almost) movement among a lot of people to Go Vegan and how this is somehow because they want to save animal life and so on. This episode is absolutely not about that. In fact, as the title says, it’s about killing animals that are to be consumed with consideration because (a) it’s a good thing to do (b) it actually makes fishes taste better (This episode specifically mentions fishes in Japan, and having eaten fish in Japan, I can confirm that fish dishes in Japan taste great even though they have less than about half the spices that a typical fish dish in India would have!)
Future Perfect - How our drinking water could help prevent suicide

Another interesting Future Perfect episode: Lithium in the water can reduce the number of people who suffer from depression! In fact, “Four of five studies reviewed found that places with higher levels of trace lithium had lower suicide rates.” There’s a bit of doubt and debate about whether this is a real thing or not. Irrespective of that, it is an interesting story and connects well to the story of the Fluoride Wars and how one person started a movement that convinced almost a whole town that putting Fluoride in the water supply is unnecessary and harmful to the residents!

This is end of post 3 in this series!

Podcast Review - Nov 2018

2018-12-16T00:00:00+00:00

This post might be about 20 days late, but it sure is packed with some of the best stuff that I heard in the last month. There’s a lot of foreign policy and politics related stuff this month.

The Impact - Plane Crash hospitals vs. Car Crash hospitals 5/5

This episode is about how every single plane crash automatically starts an investigation. The investigation eventually leads to a diagnosis of what went wrong. Using this diagnosis, the check lists are updated, new processes are introduced to ensure that the chances of the same problem causing another accident goes down signficantly. Whereas, car crashes are rampant and happen every single day. No investigations are done, and it doesn’t seem like the situation is improving. What happens when a hospital applies the Plane crash principal to solve a problem that they realize is too big? It has a lot of subtle points about authority, ego, and how a change in a process, inside any institution, can’t be overnight even if it has provably better results.

The Impact - The Opiod Crisis

There are several podcasts out there about the Opiod crisis in the US right now. It’s a very country specific problem and is apparently on the rise in Europe too. What I find interesting in this problem is how rampant it has gotten before it attracted government attention. How the problem started itself is a textbook process of a bunch of sketchy pharma industry people getting together and deciding to start a “brain washing” campaign for doctors - with all expense paid trips to conferences where they are subtly told about how they should treat pain more aggressively. The Daily did a 2 part series on the people behind the first pharma company that realized there was an opportunity here and how despite so many years of investigations into this, the people at the top escaped unscathed. A riveting story of how corporate obligations can be met without doing any good for society as a whole.
The Daily - What the West got wrong about China, parts 1 and 2 5/5

Another really informative series about the basic assumptions about China that the West got wrong: a restricted economy would never prosper. The West has always had this obsession with the axiom that “Democracy is the only system that succeeds” and now, more and more countries are threatening to prove that axiom wrong. As scary as that is, analyzing how exactly this happened is probably our best way forward. Another very interesting part of the series is where everyone believed that once the Internet becomes more popular, it will be impossible for the Chinese government to curb dissent. Of course, The Great Firewall came up and instead of the Internet being a liberating tool, it was used by the government to spread propaganda!
The Daily - What Facebook knew and tried to hide

A short introduction to the decision making process in conglomerates as big as Facebook where one person never holds the key and one person can never comprehend the size of the operation. The story is framed as some sort of an ideological battle between Zuckerberg and Sandberg. I am not sure where that comes from, but it definitely casts Zuckerberg in the same light as the Social Network movie - the college student who is very brainy but still doesn’t understand the business behind Facebook. I think that is far-fetched, but nonetheless, this is an interesting listen.
Everything is Alive - Ana, the Elevator

A new podcast where the host interviews someone playing an inanimate part of our lives. It’s weird and cooookie to begin with, but it definitely grows on you. This episode was interesting to me because I recently started working in a building where there are about 60 elevators split over 5 elevator lobbies. These elevators are going up and down every second of the day, ever since they were installed. And they are privy to so much of the people in that building’s habits and conversations. (Not too many conversations because Japanese manners frown on talking in public places - trains, elevators)
Everything is Alive - Tara, the Bar of soap

My interest in this episode is again personal. Moving from India to Japan, I had to give up soap bars and start using Body wash! Thankfully, I was able to find body wash of the same brand as I was used to in India. Nonetheless, this has a very interesting and confused narration from the Bar of soap where she talks about how the less she is being used, the higher her life expectancy is. So, while she seems to feel bad that she isn’t being used as much anymore, she’s happy she gets to live longer!
Today Explained - Do It for the Gram

A scientific one: this is about how the Kilogram was recently redefined to be absolutely measurable by anyone - with enough money, space and motivation to actually want that kind of accuracy. Earlier, the definition followed the simpler path of having a block that was defined to be 1 kg. This was again one of those awe-inspirin moments when I realized that we have come such a long way in these fields where we clearly understand that working together with everyone in the world is the only way to solve a problem. This time, the problem being: standardized measurement.
Ezra Klein Show - Doris Kearns Goodwin

A presidential historian. Given how much of the media cycle and brain space American presidents take up in the form of news, movies, podcasts; it’s only fair to listen to someone who has spent her time curating stories about these people at the very top of civilization and yet somehow very hard to understand because, as she begins the podcast with, “You have to be a little crazy to want that kind of pressure”.
Ezra Klein Show - Jonathan Haidt 5/5

And finally, the podcast that I want everyone to hear. I was fairly impressed with this conversation. Impressed enough to buy his book the next day and read it through the weekend. It was one of the most eye-opening books for me. Though I have known for a while that people don’t change their political beliefs no matter how much logical arguments you throw at them, this was the first book that really spelled out why that was in unambiguous language. There’s a lot to decode in the book and I intend to write a review and a blog post about that. The podcast is about the recent campus movements and where they came from. There’s this one section of the podcast where they discuss how kids nowadays have to be forced to go outside and leave their computers and how, in a weird way, it makes them the most well-behaved children in the past few decades. All they want to do is stay at home and chat on the Internet. The podcast is full of revelatory moments like that.

This is end of post 2 in this series!

Podcast Review - Oct 2018

2018-10-27T00:00:00+00:00

Note: The only series on this blog is still the 100 days of writing I did last year. I keep starting series and never writing the second post in that series. I really do hope this one is different. Specially because I want to keep track of the great podcasts that I listen to over the years. In fact, just to ensure that I get this done, I am adding this to my calendar. It’s a monthly 2 day event (from 27th-29th). I am repeating it for the next 1 year, we will see how I do then.

I have been listening to a lot more podcasts lately (Courtesy: 30 minute commute to work and random walks around the streets of Tokyo). And I have actually heard a lot more great content that I can believe! It has just been one great episode after another. Seriously. I am not even kidding. I started listening to the serious Ezra Klein Show at a point where I had run out of Daily (NYT) episodes. The Daily (NYT) has a huge list of episodes that I absolutely loved! Here are a few episodes that I highly recommend:

The Daily - The War Inside the Catholic Church

This is the epic fight between the people who like the current pope (for his progressive policies) and a faction inside the church that doesn’t. While the sexual harassment revelations last month shook the church from the inside, this small faction activated and started working against the Pope in this moment when he is the most vulnerable. Of course, this reduces the value of the Church as an institution, but always remember Mitchell Garabedian’s line from Spotlight (2015): “The church thinks in centuries, Mr. Rezendes. Do you think you have the resources to take them on?” Remembering this line as I heard this podcast and realized what was going on, really sent chills down my spine. (It was also about 18 degrees celsius that day)
The Daily - Lost in the Storm, Parts 1 and 2

The heart-breaking story of a family stuck in the middle of a natural disaster without the right help from the government and local institutions. This was absolutely first-class reporting, done from as objective a point-of-view as one can be expected to have when covering a story like this. Michael Barbaro and Annie Brown: kudos :applause:
The Daily - How the Opiod crisis started

As mind boggling as it is, the opiod crisis was started by a couple of people who wanted to make a lot of money quickly. YES. It’s not something that just happened. They made the drug; they sent the medical reps across the US to convince and coerce doctors into prescribing these medicines; That’s how the Opiod crisis started. This episode gave me a really deep look into how much of our lives is controlled by Big Pharma.
The Daily - Roe v. Wade, Parts 1 and 2

Again, a landmark issue in politics. These two episodes track the life and troubles of Jane Roe, the iconic woman who chose to remain anonymous while trying to get an abortion. She isn’t clear about what she wants, she flipped, she changed her point of view. “In the end, she was’t claimed by either side” - this might just be the saddest way her story ended. (Interesting even if you are not interested in the politics of this issue)
The Daily - How Trump Really Got Rich 5/5

What can I even say? I was left gaping at a blank white wall at the end of this episode. Everything was explained extremely well; and yet, I ended up with just one question: How did they think this up?
The Daily - What Migrants Are Fleeing

This connects to another Ezra Klein Show episode that I have heard recently. I won’t recommend that episode because the discussion was incoherent and didn’t lead anywhere. Ezra Klein himself had some really good quotes in the show and that was probably the best part of the episode. I might just pick the quotes I liked and post them on Twitter (will update the thread here if I do that).

Anyway, to understand immigration, it’s important to understand what’s going on in their home countries that makes them go to a new country where they don’t know anyone. What does it take to force you to leave everything you know and are familiar with, to go to a new country, without any kind of plan?

P.S. You should skip this one if you don’t follow this so closely.
The Truth - The Off Season series - Parts 1 through 4 5/5

It’s like a movie, even though it’s only sound, it’s like a movie. An amazing story, told through a gripping 4-part narrative with amazing sound effects (one of the sound effects is the sound of an egg being broken and poured into a bowl for making an omelette. YEAH.)
The Ezra Klein Show - Why online politics gets so extreme so fast 5/5

The YouTube algorithm has only one goal: to keep users on the site. If it figures out that the only way to reliably keep people from leaving is to show them something beyond the edge, something edgier than what they are used to, it will do that. The problem with edges is that stories involving them generally end with someone falling down a cliff.
The Ezra Klein Show - Jay Rosen is pessimistic about the media. So am I.

Another essential episode for understanding what the current state of the media is. Especially, to people who have been in it for a while (Ezra Klein has been a journalist for 15 years). His and Rosen’s take on this issue are radical and they come at it from a few different angles. It’s not all sunshine and roses. In fact, it’s very dark and gloomy at 12 noon. On the brighter side, Rosen and Klein discuss the reasons for this darkness pointedly and lucidly. No fancy words here: just the plain, old truth told in simple English.

That’s it for entry 1 of this series. (Maybe the only way to keep a series up is to not call them “series”-es in the first place. WAIT, WHAT?)

Moving to Japan

2018-10-20T00:00:00+00:00

So, it’s been a long time since I talked about what’s going on in my life. I moved to Tokyo in September. I came here in the third week of September and started working at Mercari from the 1st week of October.

Mercari is a C2C marketplace (mainly for second hand goods) in Japan. There’s a Mercari product in the US and UK as well. I got the job in IIT Kharagpur’s placements in December 2017. Yes, I have been fairly silent on this blog and everywhere else about the job; primarily because there were a lot of things that needed to all go well for me to actually start working here. Getting the visa was one of the main components of that.

In the end, things did turn out well! The visa part of the process actually ended up being the least stressful! It’s rather incredible when professionals are planning your trip to a country, compared to when you have to do all the planning yourself! (In fact, at the Indian immigration counter, I was asked just one question: “What is the purpose of your visit?”!)

Now, it’s been three weeks since I joined Mercari. I have finally been assigned to a team. I don’t have an assignment just yet; I need to be on-boarded onto the team I am on before I get that. The combination of infrastructure and third party services that Mercari uses is mind-bloggingly HUGE! My closest experience to this was at Elanic where we had a single Digital Ocean server with a docker container for the API and MongoDB and some other containers. We had a dev environment that could pretty much be setup on your computer after building a few docker images. So, this is pretty overwhelming right now.

I will probably get more used to it as I run into the potholes that people generally run into when they are being on-boarded into a new team. I am also looking forward to working on something that is being used by 100s of thousands of people! That’s also something I haven’t done before. The gist: A lot of firsts here!

Apart from work, just the experience of living in Japan has been great! It’s a lot of things, but if I was forced to condense all of that into a single word, it would be: CONVENIENT. Stores and restaurants are open 24 hours; The subway runs late; You can get anywhere in Tokyo after a short train ride; The food has very little in the way of things that are bad for your health: oil, spices. Of course, the last thing about food can be a con for people who like oily, spicy food!

A huge reason for moving here, for me, was to experience living in a foreign country! Food forms a big part of that and I am only getting started on trying Japanese food. I have tried Sushi, Ramen, Soba and Teishoku meals in a variety of places with a variety of items in them till now. I definitely want to get used to Japanese food! I love everything I have tried till now, that’s certainly VERY encouraging!

As I continue to settle in here, I will keep making blog posts about the things I used to write about before. There might be more content that’s Japan-related too though, I don’t think I can quite help that!

Offline Advertising - Familiarity

2018-09-29T00:00:00+00:00

I read a few excellent (albeit old) articles on advertising recently. A few of them were on The Doc Searls Weblog. My main take away from these articles was that offline advertising is not ineffective because an ad run on TV is seen by all sorts of people and is targeted at a cross section of the population; In fact, it is very effective because of that very property.

I recently made a cross-country move (more on this in a later post). I wanted to select a phone contract in the new country. There were several companies and it was all very confusing. I had two references: a contract that my company’s HR people recommended and companies that I had heard of before (particularly, SoftBank and Docomo). I had heard of these companies not because I had used them before. I had heard of them through old-school advertising!

We all know the names of a lot of companies whose products we have never used. That doesn’t mean that the money the companies spent on getting us to know their names was wasted, OH NO! It only means that the return is more long term than that.

In terms of signaling theory in economics (for which Michael Spence won a Nobel prize), “the firm signals the quality of its product to consumers by its willingness to spend money on advertising.” (N. Gregory Mankiw: Principles of Economics, p. 401.)

– from Rethinking John Wanamaker (I highly recommend this article)

Now, when I walk into the huge super market aisle of moisturizers in a foreign country, I will see a lot of brands on the shelf. But the moment I see Pantene or Nivea, my brain will make a quick connection and this familiarity will make it very likely that I buy their products. All this pretty unscientific and anecdotal; but I am fairly confident that this is how all of us work.

I have never used Pantene’s soaps or Nivea’s creams before, but I have seen their ads on TV, I have seen big celebrities appear in their ads, so I know that Pantene spends a whole lot of money on their ads. They probably believe that their product is genuinely good, or why else would they do that? They could just get a Google AdWords account and try targetting people whose last search on Amazon was “moisturizer”.

Why I include Bootstrap in every project

2018-09-01T00:00:00+00:00

Here’s some unsolicited advice: Always start any project with the inclusion of Bootstrap. There are a few reasons I insist on this. I will go into them in this blog post. There are many such blog posts on the Interweb. I figured that adding my own point of view on this matter will not hurt anyone.

Before we begin

I had a really hard time understanding what Bootstrap really was. I had some questions in mind: Is it a library? Is it a set of CSS classes? Is it jQuery, but better? Why is jQuery required if it is only a set of CSS classes? I have figured out a very simple way to explain what Bootstrap is:

Bootstrap is a set of 2 components. 

Bootstrap CSS is a set of CSS classes that
can be used in any application. It does some great things by default (like using
a good font, using margins and paddings on the `body` tag). It has 2 major
features which are worth using in any app that aims for a simplistic polished
look: the grid system and the classes that make forms render better

Bootstrap JS is a library built on top of jQuery that provides some more
functionality. 2 of the best features of Bootstrap JS are: tooltips and
modals.

One more thing. Bootstrap has really good documentation. The documentation is really really good. It was good already. And then they added the search feature with Bootstrap v4’s release. It became GREAT.

Reasons

These are in decreasing order of significance, as I have experienced in my frontend development.

Bootstrap makes your website responsive. i.e. your app will look better on every screen no matter it’s size and shape. Of course, you should use it properly. But it is extremely hard to use bootstrap badly enough that it will look bad somehow.
Bootstrap’s Grid System. This is one of the most useful things that I have EVER learnt in frontend development. Every line of HTML I write now is written in the context of col-md-*s and so on. Learn this grid system and you will forever lose your fear of center aligning text in HTML.
Bootstrap makes your site look better. The new font, the sane defaults for margins around various elements. These things have become standards across the web and including a CSS library which will take care of this is a huge load off your shoulder.
Bootstrap makes your forms easier to understand and fill. The rounded corners, the highlighting when a particular text field is selected, the amazing UI bump that it gives to select lists. All of these things can’t be done with custom CSS. Stand on the shoulders of giants
Bootstrap’s tables will finally make tables better on the web. 3 classes that I always use with a table are the default table, table-hover and table-striped. These three classes make the table resemble an Excel sheet to some extent and actually make the app easier to comprehend. Tables are a very powerful way to show information, perhaps second only to bar charts. Bootstrap makes it easier for developers to make more legible tables for their users.
Bootstrap’s buttons convey information. Almost everything in Bootstrap comes in a set of colors, depending on the “alert” level. This color is extremely useful when developing dashboard type applications. It’s also pretty useful when developing normal applications that tend to have destructive actions (like deleting your account or deleting a cutout). I used this in a former intern where I was upgrading a dashboard application written using an Angular library and the buttons were so small that it was only the colors that any user even looked at. That’s how helpful this is!
Bootstrap makes your blockquote look better. Blockquote tags are again great ways to convey information especially when writing blog posts or some sort of a reference. Goodreads doesn’t use bootstrap and their blockqotes look really bad. Blockquotes look like text that has been indented, instead of looking like quoted text.
Bonus Bootstrap gives you the ability to add a navbar quickly. The Bootstrap navbar is extremely toned down. I don’t like it in all settings, but for some apps where you can’t do away with the navbar, Bootstrap is the way to go. The navbar has a set of options that enable you to do anything with it: stick it to the bottom of the page, show only one option on mobile screens, anything you want really.
Bonus Bootstrap Modals! I mean, this is something everyone loves. I don’t like modals because they are very hard to use on mobile and I tend to stay away from them as much as I can. I would rather enable the user to scroll through the page rather than have to struggle to get to the small X button at the top of a modal. But if you are developing an application that’s used only on desktop computers (like a Sales Dashboard), modals are extremely powerful tools to retain context and let the user edit information at the same time.

My Values

2018-08-18T00:00:00+00:00

My friend Kshitij Saraogi just released his website’s code on GitHub and I happened to visit it. His website had a very intriguing tab: VALUES. It immediately led me to question myself what my values were? What were the things that helped me navigate through decision making and just life in general?

I intend to list out a few of my values here. This list is bound to grow and I think I will keep adding / removing things from here.

Last updated 18th August, 2018

Be decisive, without being impulsive.
Lower your expectations.
Do the right thing. Your experience will tell you what the right thing to do in every situation is, just follow that instinct.
Do not regret past decisions. You did what felt right in the moment, don’t second guess yourself or others in retrospect.
Don’t over-analyze the future. Go with the flow.
Try everything, within reasonable limits.
When in a dilemma, talk to more people. But take the decision yourself. Whatever it is, you are the one who must live with it.
If you are hopelessly stuck, while solving a problem: take a break. Get perspective about what is going on and how you ended up there. This won’t always help you solve the problem or even move closer to the solution, it will atleast give your brain time to recuperate and work at it’s full strength when you return.
Don’t plan a vacation/trip down to the hour. Have a rough plan in your mind, everything else tends to work itself out once you are there.
Travel far and wide. Choose unorthodox places to go to.
Read a variety of books and read constantly. Take notes when you think something is important and return to these notes in times of doubt.
If you need something and it isn’t being sold, build it.
Be organized. Clutter adds stress, you don’t need that.
Live according to the routine that suits you best. For me, this is waking up early, going to bed early, eating food at the right times and avoiding junk food whenever possible.
Don’t drink alcohol or coffee or other caffeinated drinks during travel.
Eat safe when you travel. Listen to your body.
Talk and mingle freely with people. Most people can tell you many stories about their life that you will never find out if you don’t put in the effort to go out and spend time getting to know them.
Trust yourself to be able to figure things out.
Document the present.
Talk is cheap. Do not spend too much time discussing personal choices.

Media and Expectations - Part 2 - Indian Railway Accidents

2018-08-17T00:00:00+00:00

This is part 2 of a two part series

Part 1 · Part 2

As promised at the end of Part 1, I will list the number of people who were affected by these incidents that were heavily reported on in the Indian news media to the point where I was actually wondering whether it was even safe to travel in the Railways anymore. If I had only had a look at the numbers I am going to show you, I would never have had that fear.

The full data can be found here. The sources are listed at the end of this post.

First, let’s look at the list of deaths and injuries over a period of 5 years starting 2010 and ending 2017.

YEAR	DEATHS	INJURIES
2010	222	464
2011	128	540
2012	94	89
2013	75	81
2014	69	181
2015	110	420
2016	153	382
2017	67	360
TOTAL	918	2517

That’s less than a thousand deaths and about 2500 injuries over a period of 8 years. Including the year in which there were so many reports about derailments and problems. How dangerous is travelling in the Indian Railways? One number will reassure you.

Every year, Indian Railways carries more than 8 billion passengers. Out of them, in their worst year, 222 people died. That’s an incredibly small number.

For comparison, WHO estimated in 2013 that about 207,000 people die each year because of road accidents in India. Put another way, an estimated 17 people die among 100,000 inhabitants each year. That is way way more than the total number of people who die in railway accidents.

So, trains are far safer than road travel. And air travel just had the safest year, last year. So, flights are safer than trains. There’s no need to feel panicked by the numbers that are being shown on TV every time a train derails.

Every step should be taken to bring this number down to zero. But realize that this is a safe mode of transport. That’s the take-away here.

My argument is that these numbers should have been part of every single story on TV about this topic. Isolated numbers make no sense to a viewer. When you show them the same numbers with context and a total that they can divide it by, they will realize how important the number is. I probably don’t need to explain this, but I want to be sure I am understood here. Say there is a fictional elevator and you want to find out how dangerous it is to get on it. You know this:

On average, 2 people have gotten injured every single week for the past 8 weeks

Now, if the elevator is used by only 10 people in a week, that’s very alarming. You should probably not get on the elevator and take the stairs.

But if it’s a big building where several people work and it’s used nearly 100,000 times or more, you don’t need to worry.

In either case, it’s very important that you find the problem and fix it. But institutions tend not to be very fast about solving some problems that are logistically complicated. In that case, you need to be able to make a realistic estimate of the danger you will be in when you make a decision. To do that, you need all the facts.

So, everyone who reports news, please please please mention the context next time you report a number. Mention the total, compare it to a similar number, help your audience understand the number you are reporting. You are doing a job that’s more important than the ratings your show gets.

P.S. Do watch the TV show The Newsroom (2012). It’s a really great story about what the business of reporting news is really about.

Media and Expectations - Part 1 - TV v. Internet

2018-07-30T00:00:00+00:00

This is part 1 of a two part series

Part 1 · Part 2

I am fresh off the Factfulness dose. The authors articulate one thing about the news media:

I cannot even see the highest-quality news outlets conveying a neutral and
non-dramatic representative picture of the world, as statistics agencies do. It
would be correct but just too boring. We shouldn't expect the media to move very
far in that direction. Instead it is up to us as consumers to learn how to
consume the news more factfully, and to realize that the news is not very useful
for understanding the world

This got me to thinking about how much I have complained about the Indian news media in the past: the unnecessary sensationalization of stories, the shouting, the talking head shows that run throughout the day with a macho news anchor who is shouting the populist opinion (whatever that might be), generalizations. I realized that they were pandering to every single instinct mentioned in Factfulness. A quick summary:

The gap enables the powerful people to hide information and fool the majority of people who are not powerful
Negative news stories only: deaths, harassment, lynchings, bans
3 bad things happened last month? Expect 6 more this month, 9 the next month and so on! Things will only go downhill from here!
Be afraid of the powerful people and the authority figures because you might be the next victim. Don’t let them tell you how to live.
Blow things wayyyy out of proportion
Generalize everything across state identities, religions, nationalities, communities
Bad things happen in this state or the other and have for years, it’s stupid to expect change from the people in power because they want to maintain the status quo
There’s only one view of every single story: the news anchor’s view
Blame the first person who is accussed and conduct a media trial immediately and declare that the person is guilty
You MUST ACT NOW! Tweet us, use this hashtag, sms us, call in and express your opinions, do it RIGHT NOW!, where else will you be heard! Time is flying by as is obvious from the 10 moving elements on your TV screen.

Next time you watch a nightly news show on Indian TV, look out for these instincts. You will find them.

This is not a rant. I am not blaming anyone here. I have done what’s in my power: I have stopped watching these news shows. I have told every single person who says they watch these programs about how they are run, about why they should stop watching these programs and instead read or watch videos on YouTube. If they are still interested in news, videos by channels like Vox and Vice News. If not, there are enough cat videos to fill your heart to the brim.

Truth: There are always enough cat videos.

The news business runs on advertisements. Shows that get eyeballs, get ads and make money. Every segment should keep the viewer glued to the screen, it should make them want more, it should ensure that they don’t reach for that remote. So, the first thing we need to do is to stop blaming news channels for the things they choose to run. It’s what the consumers watch, it’s what they get ads for. It’s not their fault, it’s the business model.

It’s like that Zach Holman talk[FN1] about the HR department in every company. The HR department works keeping the company’s interests in mind. Similarly, the news media works keeping the advertisers interests in mind, not the consumers. As soon as you understand that, you can start correcting for it. You can understand why a particular story is being run. You can understand the other side of the story, what the news producer was thinking when they approved the script for the opening lines by the news anchor.

Now, the YouTube channels. I can vouch for only two: Vox and Vice News. They put out videos that help you understand the world. The content that Vox puts out is amazing! It’s backed by facts and is solid to the core. There is no sensationalism. There are no unnecessary graphics, in fact, the graphics are kept to the absolute minimum. The graphics help you understand the video and don’t distract you. Vox doesn’t ask you to tweet to them, to use a hashtag. They don’t ask you to do something, they simply explain the things that are. What’s the advantages that YouTube has over TV?

People don’t have to watch: They have a lot of content to choose from. TV on the other hand is limited. During prime time, 9 pm to 10 pm, all Tamil TV channels run bad reality shows. If you are not into sports and don’t want to watch a movie along with your dinner, your only option are news channels.
There is no primetime: Your content is out there, forever. It can be watched all over the world. It can be reviewed, people can call you out if you made a mistake. The bad part is that the people who watch your videos will also see your mistake. But TV doesn’t support criticism. You would have to criticise a TV show on the Internet and how many people who watch a show will see your YouTube video or blog post especially if you are just a normal guy?
Ad revenue is directly tied to the number of people who watch your content: On TV, it’s not the number of people who watch but the rating of your TV show. As the TV regulatory agency put it very well in a report I read and analyzed on this blog a few weeks ago, ratings decide the content that gets made and ratings are collected from a sample of the population. No one can measure exactly how many people really watched a show. They can only measure how many people in the sample watched that particular show. These are the numbers that big companies who want to run ads on these shows get. On YouTube, you get the exact number of people who watched every single video. The demographics of each video’s viewers, the country that the video is popular in, and so on! All of this extra information is a boon for advertisers who can target exactly the people they want. So, channels like Vox must make a huge variety of high quality content. That’s the reason there are so many types of Vox videos: Almanac, Atlas, the series about Pop culture, the series about things that are hyped up, the amazing Emmy nominated Vox Borders!
Diverse and prolonged viewership: TV shows are available for one hour a day to 1 billion people. There are no re-runs of talking head shows, there is no value in re-running them because the information that they were trying to dissipate is old the minute after the show aired for the first time. Producers don’t have to make the show re-watchable, they just have to make the show sensational enough to make people watch it once. And not “people”, just the people who are part of the sample which contributes to the ratings. I don’t think there’s any way to actually influence the sample but not everyone else, still, it’s one less thing to worry about. Vox video producers on the other hand have to worry about making the video watchable, shareable. There is nothing better than a personal, trusted recommendation. They have to make the video good enough to stick around forever and be re-watchable by a ton of people around the world. That’s why most Vox videos have closed captions in a few languages, that’s why the voice over is never messy or unclear, that’s why the graphics are never distracting. (That is my assumption)

Said in a sentence: Businesses do what they have to do to excel at their business model. Both TV shows and YouTube content creators need to get as many eyeballs as they can. But the set of constraints is different. TV shows need to be appealing for a short period of time, they needn’t have re-watch value, and they need to attract the people in the TRP sample. YouTube content creators need to make content that’s worth watching atleast once and preferably several times, the content should be shareable and should reflect the type of all content on that channel, it should be true because people can call you out on your mistakes and your viewers are right there watching out for such call outs. They each work to excel at their business model.

This was part 1 of a 2 part series on Media and Expectations - TV v. Internet

Part 2 will cover a particular case of Factfulness. A short synopsis:

Most Indians will remember the stories about train derailments during the months of May 2017 to August 2017, there were nearly 9 accidents. There was a huge uproar on the over-use of Indian Railways infrastructure and several thousands of column inches and TV show hours were spent debating and discussing this. The problem with each show was the same, they presented one number: the number of accidens, deaths and injuries that had happened in the past 3 months. There was no number to compare it to. In part 2, I will try to figure out what the numbers really are and then compare them to older numbers. I picked this problem at random because it affected the way I think about the safety of trains. I don’t know what the outcome will be yet, I am still collating data. Either way, I think it will be a worthwhile experience.

FN1: The talk is called Getting Fired and it’s a great way to spend 45 minutes of your time.

Pune Diaries 1 - Art Gallery

2018-07-26T00:00:00+00:00

For the last 5 years, I lived in Kharagpur, West Bengal. It is a town, no more no less. There’s nothing to expect there, and it lives up to that expectation. A movie theater started flourishing in my last year there. Apart from that, there was nothing to do there.

After those 5 years, and even in between during Summer and Winter vacations, I was in various cities: Chennai, Bangalore, Coimbatore briefly. But I never did anything that I was unhappy about being unable to do when I was in Kharagpur.

Now, I am in Pune until the first week of September. I am in the training period for a job, and my weekdays are not free in any sense of the word. My weekends though, are free and I would like to use them well. And hence, this is a new series on this blog which will follow me around this city as I do the things I was unable to do in a town like Kharagpur. Some weeks this stuff is bound to be boring, but I will try to keep it as interesting as I can!

Last weekend, I visited the Art2Day Gallery in Pune to see the exhibition The Power of Point. The exhibition is a collection of ball point pen drawings by Sheil Sadwelkar. I read about it first in the Pune Mirror. The thing that intrigued me most about this form of art was that ball point pens are notoriously difficult to erase. In pencil sketches, and even in paintings, there is some margin for error. But ball point pen drawings completely remove this margin for error. The drawing must be right the first time, or you must start all over again. That is a great sort of finality, and I wanted to see what the subjects of these drawings were and how they affected me when I saw them.

(Note: I am not an art buff. This caught my eye and I decided to go.)

The walk from the place I stay to Art2Day was pleasant and took me through some quiet parts of Pune. It was about 4 pm and there was a slight drizzle. I went right in front of the Brihan Maharashtra College of Commerce’s front gate (I think it was the front gate)

The drawings were really good! I was completely taken aback by the detail and intricacy in each of the paintings. Three paintings really caught my attention.

The Final Destination

This particular painting had a road that wasn’t visible after some distance. It disappeared behind a mountain of some sort. This appealed to me as such an appropriate analogy for daily life! Everyday, the problems that I face in that day, week or month seem like the largest I have ever faced and will ever face. A few months down the line, everything seems to make a lot more sense. In general, it’s like that famous quote: “Life can only be understood in retrospect, but you must live it in the other direction”

I was unable to find an image of this painting!

Plastic Bonsai, Candyfloss River

Throughout several of the drawings, the common thread is the use of parallel lines to suggest density. For example, the ground is densely packed with vertical lines in the background, and over that the subject is drawn, roots in these paintings with trees. And as the ground reaches sky, the densite of these vertical lines reduces and suggest the beginning of air. This is absolutely mesmerizing to watch, because the first time I saw these drawings I didn’t even notice these in the background. Only on close observation did I realize that the effect of these lines was perceived automatically and there was no conscious effort required to think about them! FREAKING AMAZING!

Eyes One, Eyes at Visarajan

Another trick that pervades all of the art art these small curves that seem to be connected. They are clearly visible in any of the paintings with trees where the density of leaves and the suggestion of being hollow or filled is shown clearly using the density and the closeness of these curls. In some places they are small and dense, whereas in other places they are farther apart and create the near-solid background in some drawings.

Apart from the paintings mentioned above, I especially liked Fuji Bypass and Fallen in Love as well.

The exhibition ended today. So, I can’t really tell you to go see the paintings this time. The exhibtion is sure to continue somewhere else. If you are ever in the city where these paintings are being shown, definitely find a way to see them! It was a great experience!

P.S. Some of the paintings can be found online here.

WEEK 1 OF PUNE DIARIES IS OVER

Donna Tartt

2018-07-03T00:00:00+00:00

Primer: Donna Tartt is a 53-year old writer from Mississippi. She has written only three novels and she released them in 1992, 2002 and 2013. Each of those novels is nearly 600 pages long. She rarely gives any interviews.

This was one of my major goals for this summer. I had already read The Secret History by Tartt and after reading that book, I seriously believed that she was an author unlike anyone I had ever read. Some thoughts from right after I finished the book:

An attrocious act, on the face of it, but surprisingly towards the end, I was
empathising with their situation, and although, I feel that this may not have
been the best course, I can't question the decision that they made to kill their
friend. That, only because of some really involved and amazing writing!

This was in December 2015, nearly 30 months ago. Both her other books: The Goldfinch and The Little Friend are 700 pages long and I have found some great excuses in these 30 months to never pick them up.

This summer (the first after finishing college), I was determined to finish one of these books. I was confident that if I didn’t finish this book right now, and tried to pick it up after I started working, I would almost certainly never read this book. That wasn’t acceptable to me.

Anyway, after reading The Goldfinch, I realized that Tartt is really something. I couldn’t let her third and only book I hadn’t read left unread. So, once again I started reading her third and final novel. Now, I have read all three of her books and I can’t be more happy that I decided to spend this time on her books because they have been a lot more than just another novel.

I can’t choose a favourite Tartt novel (I know, it’s cliché to say that). It’s because I look at her work as a whole. All three of her novels (and anything else she releases later) seem to be a part of the same treatise: A treatise about life, drugs, beliefs, people and unused time.

In all three novels, the inciting event is a fairly content protagonist who does something unconventional. Something that they can’t or aren’t willing to explain to the reader. Why did Richard change his major to the classics? Why did Theo take the painting out of the museum? Why did Harriet feel so obsessed over finding out the real story behind Robin’s death? I like to think that they did these things because they were meandering, they didn’t know where they were going and needed to do something, good or bad, that would set in concrete the path they would set out on.

The stories of Richard, Theo and Harriet go through a lot of stunning twists and turns. One doesn’t believe a 12 year old capable of stealing a snake and attempting to kill someone. One doesn’t believe someone could steal a 400 year old painting from a museum and never see it once, for a whole decade! There is no lowest common multiple that binds these stories together.

But somehow, they still feel connected.

Landing pages on TV

2018-05-30T00:00:00+00:00

One fine Monday evening, I was absent mindedly scrolling through Twitter, which was when I came across this tweet from TRAI, the Telecom Regulator in India:

Stakeholders sharing their views on Landing Page Issues in Open House Discussion on Consultation Paper on 'Issues related to placement of TV channel on landing page' at TRAI HQ, New Delhi @TRAI pic.twitter.com/6ZQPKMkitk
— TRAI (@TRAI) May 28, 2018

Understandably, I was confused. Mainly, I wasn’t sure this topic needed such a formal open house discussion which no doubt made executives from the stakeholder companies fly to Delhi to sit in and make sure their voices were heard. So, I retweeted this and forgot about it. A few minutes later, Abinash replied to my tweet and set of a chain of events that had a very agreeable ending!

What? Did you make sense of it later? :P
— Abinash Meher (@beenharsh) May 28, 2018

So, when you turn on the Set-Top Box, there is a channel that shows up. This channel is the landing page of the STB and apparently, STB distributors (think, Tata Sky) and broadcasters (think, Sony) were eager to put content that they wanted their customers to watch on this channel. This is a great marketing strategy, and TRAI wouldn’t necessarily come in to stop that if it was just that. There is actually a slightly more impactful effect if a distributor gets a particular channel onto their landing channel: it’s rating increases.

To understand this, we need to understand how TRP is measured. There are meters that are installed in particular houses which are selected to have a good mix of demography (age-group, gender), viewing time, etc. The TRP meter, the Bar-O-meter, captures the TV’s audio. Every channel’s audio has a fingerprint that probably has information related to what channel it is. So, the meter can capture which channel the user is watching.

Another slightly more delicate detail is that the measuring is done at the resolution of clock minutes. So, whichever channel is viewed for more than 30 seconds in a given clock minute gets the credit for that complete clock minute.

Now, when the TV is switched on, and say, NDTV is the landing channel, the Bar-O-Meter will start picking up that the user is watching NDTV. If the viewing time in this particular clock minute exceeds 30 seconds, then NDTV will get a minute of watch time that the user didn’t really ask for. In fact, it was purely incidental and any channel in NDTV’s place would have been able to get that watch time.

Clearly, this is not desirable. The reason that TRAI gives in their consultation paper is incredibly and something to very deeply think about. It was nothing short of a revelation when I read and understood what their justification really meant. This is what it was:

1.9 With  the  growth  in  the  number  of  channels  and  increasing  variety
of programmes  available,  the  task  of  both  broadcasters  and  advertisers
in   allocating   resources   becomes   increasingly   difficult.   Mark et
segmentation and targeting have become vital in such a scenario.  On the  basis
of  audience  measurement  data,  ratings  are  assigned  to various programmes
on television. Television ratings in turn influence content  and  programmes
produced  for  the  viewers.  Better  ratings would  promote  a  programme
while  poor  ratings  will  discourage  a Page 8 of programme  or  content.
Incorrect  ratings  will  lead  to  production  of content  which  may  not  be
really  popular , while  good  content  and programmes may be left out. False
and  misleading ratings, therefore, can  affect  not  only  broadcasters  and
advertisers  but  also  the  viewing public   as   well.   Therefore,   there
is   a   need   to   have   accurate measurement    and    representative    te
levision    ratings    for    the programmes.  Placement  of  television
channels  on  multiple  LCNs obliterates   the   above   said   objective   of
the   TRP   guidelines   and measurement.

So, whenever someone turns their TV on and either waits for some time to switch channels or else looks through their programme guide trying to decide what to watch, these clock minutes will be added to the rating of the landing page channel. That’s the reason TRAI passed a direction in November 2017 asking distributors to not place any TV channel whose viewership is being measured on their landing channel.

1.11 ... After  due  consideration and deliberation with stakeholders , the
Authority issued a direction dated 8th November,  2017  wherein  it was directed
that all  the  broadcasters and  distributors  of  television channels,  to
restrain  from  placing  any registered  satellite  television channel,  whose
TV  rating  is  released  by TV  rating  agency,  on  the landing  LCN  or
landing  channel  or  boot  up screen.

Full text of this direction

This direction was apparently only a temporary solution, and TRAI has now set out to solve this once and for all. I do admire the structured methodology with which they have set out to solve a problem that appears to be rather trivial at first glance, but is everything but!

Auxilliary

There is the issue of updating technology and how it is handled by a bureaucratic organization such as TRAI. This is an interesting study of what happens when technology moves forward faster than the law was able to.

So, in 2004, they released the first Interconnection Regulations, which dealt with how distributors should inter operate with each other when they transmit analog signals (Cable TV).

Then, in 2012, they released the second Interconnection Regulations, which dealt with digital addressable systems that are being used now. The digitization of the TV services sector was started in June 2012 and austensibly completed in March 2017.

After this completion of digitization, TRAI notified everyone that Interconnection Regulations 2017 is to be followed now and that the 2004 and 2012 versions have been repealed.

But, this is where the twist is. DUH DUH DUUUUUH These 2017 regulations are sub-judice in many courts across the country! So, you can’t apply them just yet. TRAI has to wait for all the court cases to end and only after that can they apply these regulations. Until then, the old regulations (2004 and 2012) will remain in force!

And it’s entirely possible that by the time these 2017 regulations can be applied, the next TV technology is already out there and becoming popular and the whole cycle will start over again.

Reading material

The Consultation Paper

Particularly, you should definitely read the sections 2.9-2.11 which detail how TRP is measured. It was very educative to finally understand how TRP was measured, although I still don’t know how they go from the measured clock minutes to the actual TRP ratings.
The Open House discussion notification

TL; DR

When you turn on your Set-Top Box, a default channel will show up on your TV. This is called the landing channel or the landing page of the TV
When not regulated, landing channels can be used by distributors and broadcasters, to artificically, increase the ratings of their channels; this is a direct consequence of the methodology used to measure TRP ratings
Inaccurate TRP ratings are bad because when there are so many channels, TV ratings determine what sort of content is produced by TV channels
Hence, we should ensure that what appears on the landing page should be something that makes the experience of the customer better, without having any ability of favorably or unfavorably affecting the TRP ratings of any other channel.

THAT was the reason for the consultation paper, the consultation process and the subsequent open house discussion. I hope it makes sense now.

OpenSSL is extensive and amazing

2018-05-25T00:00:00+00:00

Back in August 2017, I hadn’t yet dipped my feet into OpenSSL’s command line yet. That’s when I ended up finding out about a set of OpenSSL commands that have a solution for most cryptography related requirements. Secondly, it’s a great way to just explore and find out about new technologies. So, here’s a list of commands that absolutely floored me when I first found out about them:

1. openssl x509

$ openssl x509 -in ~/Downloads/twittercom.crt -text | less

This command inspects x509 certificates from the internet. Firefox has a great certificate inspector, but for every field, you have to go and highlight it and then the value for that field will be shown in the bottom. This on the other hand shows up in gorgeous less and works like a charm! I have learned so much about TLS just reading the certificates for the top sites that I keep visiting.

For eg: Basic Constraints and Moxie’s talk about defeating SSL using it.

2. openssl rand

$ openssl rand -hex 40
$ openssl rand -base64 40

How many times have you wanted to generate some random bytes quickly? OpenSSL can do it! I don’t know what the distribution of this is, or what it uses as a seed or any of that, but it just seems to work and that’s great!

3. openssl s_client

openssl s_client -connect duckduckgo.com:443 -tls1_2 -showcerts

This opens a TLS 1.2 tunnel with the destination site and then allows you to send some data on this newly created encrypted tunnel. This command also shows all the options explicitly and lets you inspect each handshake. This was pretty helpful when I was writing tls-handshake.

4. openssl enc

# the command to encrypt a file
$ openssl enc -in 404.html -out 404.html.enc -k "testing123" -aes-256-ctr
# the command to decrypt the encrypted file
$ openssl enc -in 404.html.encrypted -out 404.html.new -k "testing123" -aes-356-ctr -d

This command comes in pretty handy if you want to send a huge file to someone using something like Google Drive or Dropbox (a vendor that you don’t trust to not look into your data). Put everything in a tarball, encrypt it using AES, then send the encryption key to the recipient using another channel (such as Signal) and the encrypted data using the cloud storage service. I haven’t really gone into it’s performance aspects, but it’s a pretty cool command.

Of course, you could use the same cloud storage provider to share the key if you have the public key of the recipient. You could use something like PGP or simple SSH RSA keys (if they are long enough).

Bonus: You can use the openssl rand function to generate a secure key that’s a few 100 bytes long and can’t be brute forced for the forseeable future.

One of the drawbacks of openssl enc is that AEAD ciphers aren’t supported in the version of OpenSSL I have (1.0.2g 1 March 2016) (eg: aes-256-gcm). There is some discussion on github about this. Also, this PR that says enc will not support AEAD in the future. It says to look into the cms utility.

The Futility of Surprise Tests

2018-04-14T00:00:00+00:00

P.S. I had my masters thesis presentation today. I started writing this on 14th April, but of course I didn’t get around to publishing it. After a month and a half without a post, I don’t even think I can make a proper excuse. But I am done with college, and I am going to be free till the second week of July and there will be regular posts until then. (I am thinking twice a week. Forced-sort, so that I don’t have the choice to not write. Kind of like a short re-run of 100 days of writing. Enough blabbering, on to the post now)

Anyone who has been a student in India for more than 10 years will know that surprise quizzes are the oldest tool in the teacher’s arsenal. It is wielded whenever a teacher feels that they are not being given due respect or the students are taking the course too lightly or (and this is the best) when a teacher feels that their students are not studying regularly. How sharp is this tool? And consequently, how far into the bulletproof vest of a not up-to-date student does it enter?

Anyone who has been a student anywhere also knows that very few people study regularly. So, what’s the point of Surprise tests really?

Some context: A few weeks ago, a surprise test was conducted. I had sort-of flaked out on the classes and I had some notes, but I mainly didn’t know much. This was the post mid term surprise quiz, I knew the pre-mid-term stuff well but nothing after that. Surprisingly, for me, I got the correct answer for both questions! It was really incredible. That was when the question came up. If I was able to get both questions right, what did the test screen for?

There are a few things a test generally screens for. It can screen for knowledge, it can screen for ability of applying knowledge, it can screen for understanding of the origins of knowledge (subjective questions, derivations), it might also screen for nothing at all. A version of classroom smoke screen that exists for no reason.

10 days before that subject’s end term exam, I can safely say that more than 90% of the class was completely unprepared. Those who had attended classes might have some idea, those who hadn’t shouldn’t have had any idea except for what they gained here and there in the few classes they attended. A teacher who has been teaching for 6 years knows this and thus, the surprise quiz was not screening for knowledge or it’s applicability.

Was it screening for origins? Maybe. This is a tough one, because theoretically, someone with a threshold amount of knowledge should be able to complete a derivation. But if you have ever studied Real Analysis or Switching and Finite Automata, you know that most derivations follow a very specific approach. Approaching it from any other direction will lead you nowhere. (This is not true for Graph Theory, where a lot of approaches seem to work. This course wasn’t Graph Theory)

That’s about all the evidence I have. I don’t see the point of surprise quizzes at all. But apart from being useless, I think they are harmful because they reward the wrong set of people: basically, a random set of people.

Surprise quizzes are often conducted by professors who don’t take daily attendance in their classes. Instead, they keep saying “There could be a surprise test on any day”, the implication being that students should be scared enough to come to every class. Even if a student is scared, something might happen that leads to the student missing a class. I have missed at least 5 8 am classes where a surprise test was conducted in a course where I attended most of the other classes; I am talking not attending 5-6 classes in the whole 4 month semester.

So, basically, surprise tests as a proxy for attendance reward the wrong set of people. And they eliminate the reward for the people who should get it. I feel bad about getting the answers right because if the teacher had taken attendance, I wouldn’t have gotten those marks and that would have been fair.

So, laziness leads to surprise tests, and surprise tests don’t reward the people who are regular. TL;DR Surprise tests reward a random subset of the students in a class and should be done away with.

Cutouts 1.0 and the convenience of monolithic frameworks

2018-03-20T00:00:00+00:00

This graph shows the commits made in the cutouts repository over the past 1 year. I have been rather dormant the whole year on this project. In fact, the last time this project saw some heavy development was back in October 2016 when I was working on adding tags to the system and making it a better product that I could use.

The surge in development during February was right before the Metakgp Demo Day on February 15th. There were same gaping problems in Cutouts that I had never considered. And the main reason behind this was that there were no users except me. And I was using the application just as it was intended to be.

When others starting using it, they told me that the application was crashing whenever they tried to add a Cutout. The logs showed me that I was doing URI(link).host for the link variable and there were no validations setup that would ensure that users could only add URLs in that field. This was probably one of the glaring problems that I found and it acted as a trigger for me to comb through the whole application and figure out what other assumptions I had made which a non-me user would not instinctively abide by.

Adding tags was one, where new users expected tags to work differently. Another open feature request is that the link field shouldn’t be compulsory, instead the user should be allowed to add text in that field. (Reported by Kshitij who uses Cutouts to store quotes from books!)

After adding a title attribute to all cutouts, life on the application has become far more convenient. Mentioning the charset in the downloaded HTML was another minor goof that has lead to a prettier exported cutouts HTML page.

Yet another issue was with the ordering of cutouts. The default ordering throughout the app seemed to be chronological, when the reverse chronological order made a lot more sense as you would want to see the latest cutouts before your older cutouts.

At this point, I was fairly happy with the quality of the application for new users and the application had stopped crashing whenever people tried to do things that weren’t standard, instead it would either give a decent error message or fail gracefully without crashing. This was the v1.0 point for Cutouts.

A couple of other features that I wanted to implement mainly for my own benefit but that I figured would bring more users in (because of the network effect) were sharing a cutout via email and creating a permalink for every cutout. With the right meta tags, sharing them on Facebook and Twitter would cause more people to visit cutouts and hopefully become members and use the app. Using Rails Action mailers, this was far easier than implementing something similar would have been on a Node framework like Express. Creating a permalink was again simply creating a show method for the Cutout controller.

During the Demo Day, Vivek Aithal gave me the suggestion of implementing a feed of what everyone is cutting out. This was the ultimate way to use the network effect to my advantage. Once again, Rails surprised me because implementing it was incredibly easy!.

The few things where Rails wasn’t pleasant to work with were the updation of the Rails version itself (which I didn’t complete successfully, I am still on Rails 4), the updation of Bootstrap to 5 (which I completed properly, although it took an unreasonable amount of time to get there). The updation of the versions of other Gems was much simpler. The bootstrap update gave me access to the new color scheme that pops compared to the old, duller scheme.

Looking back at this revamp and relaunch, I realize that Rails has been very very helpful whenever I have had to implement a new feature. Even if the feature is involved and seems daunting at first, the pieces in Rails fail into place very well and the implementation itself is tens of lines at most. Also, once you make a commitment to using a monolithic framework like Rails for a big project like this one, there is no going back. I don’t even want to imagine what sort of effort it would take for me to transfer the application as is to another language.

Where Rails faces it’s most frustrating caveat is the management of the development environment. The different gem versions, the ruby versions, the rails asset pipeline: handling them is a huge pain, they don’t just work^TM. If you want to update a Gem, that’s a completely different battle and I wish you luck with fighting it.

Node.js on the other hand would probably need a few 100 lines to build these features, but the dependency management in Node.js is way simpler. In fact, when I went through the same process on cli-cube-timer, the whole process was a lot more pleasant and the main problem came up wherever the updated dependency had changed an API. That seemed like a better use of time than struggling with gem installations and those type of things.

I will still say that Rails is a good framework, despite the frustration that has often caused me to question my choice of framework back when I started this project. Implementing new features takes very less time and once you get the hang of it, it’s fun to write code in Ruby, although I have my own gripes with the language. I will save them for another post. Another advantage of Rails is the gem library and the community around it. There are gems to do almost anything. But the same can be said about Node.js, so this isn’t an advantage as much as it is a leveller.

Right now, there are some minor changes that are upcoming on Cutouts. The long term goal is to have an application that has good test coverage. The short-term goal is getting to v1.1. I have already missed the due by 2 days and rescheduled further to the end of March. I will get there, soon. I hugely underestimated how much I already had to accomplish in March.

P.S Cutouts now has a Twitter account: @CutoutsApp! If you use Twitter, follow @CutoutsApp to remain posted!

A Wildly Functional Digital Studio

2018-02-24T00:00:00+00:00

I follow Casey Neistat. I have watched everything he’s made, and everything he continues to make. There’s one constant character in all of his work: his studio.

He has had the space for nearly 10 years now. And everything in it is outfitted exactly as he likes it. He never searches for things in his studio, he just gets them.

Hang on, this is not just a rave about Casey Neistat’s studio

I have been thinking about what the equivalent of a studio would be for a programmer. The studio is the home for all the tools that you would use in your work and life outside of work. That has got to start with a few staples: your OS, your editor, your shell, your dotfiles. Actually, the complete dotfiles repository. But is that all? Definitely not

Scripts. There are a lot of repositories on Github with the name scripts or oneliners which have a collection of a the user’s most commonly used shell scripts. Great. So, oneliners is another part of the studio. What next?

Dotfiles and oneliners have a very low time spent building them to time spent using them ratio. So, they are a really good way to spend your time. Write more and more oneliners. As many as you probably can churn out. The ideal case would be one in which any task could be accomplished using just your brain or your oneliners readme.

What about more involved contraptions? (eg: Neistat’s Overhead shooting rig) These are not the typical build-once, use-forever shell scripts. They are more involved. Sometimes, they are even complete web applications. Most of the time though, they are single-file scripts or web applications that solve an extremely personal problem. Eg: me with cutouts (I really could not let that opportunity to plug my thing go) or some other things.

~Cutouts is a way to save articles you read on the internet and want to store for later.~

Cutouts is an application that I use to store the articles that I read on the Internet alongwith the name of the author, a memorable quote from the article and tags that let me arrange the articles. There are some more bells and whistles which I added recently mainly to tell more people about it and help fill a void for them, if they had one.

Share a cutout via email: The articles I cut-out, I always want to share them with someone. Engaging with people and talking about all the perspectives on the issue at hand is a great way to learn.
Share the permalink to a cutout: In case I am not too crazy about email and I just want to send it over an IM app to someone
Upgraded to Bootstrap 4: The new Bootstrap blue. double heart
Cutout visibility: Give me six lines written by an honest man, and I will find something in it with which to hang him Cardinal Richelieu
The Cutouts feed: I want to see how many people are using it and what they are upto. (obviously, only public cutouts are shown here)

Cutouts is a part of my digital studio. It’s a Rails applications that I have spent quite a bit of time on. It solves a problem for me. I don’t know if many other people have the same problem. I want to get this out there, so that if they do have this problem, then Cutouts would make their life a little bit easier.

The crucial difference a physical studio and a digital one is that the components of my digital studio need not be a part of just my studio, they can be a part of anyone’s studio.

Postscript

This is equivalent to a single shot showing all the things in a physical studio.

Component	Purpose
dotfiles	Install several dependencies, setup zsh’s look and feel, editor (vim), git, tmux
oneliners	A list of oneliners that have proved to be useful time and again and have a tendency of being hard to find whenever the bash history disappears and these don’t appear in autosuggestions anymore
cutouts	Store articles with quotes and author names. Companion firefox extension
terminal-wallet	Manage my wallet from the terminal; do simple debit, credit operations; have a stash account; export to csv
cstimer_analyze_cli	Analyze past times and note how fast improvements have come and if there have been any drastic jumps
cstimer txt to cstimer json	More struggles with csTimer in which the import and export formats were different(??)
cli-cube-timer	Measure the time it takes me to solve the Rubik’s cube; plot it over time; maintain a backup on a Github Gist (_Finally I was done with csTimer)
check PNR status	Check Indian Railways PNR to see if their status has gone from Waitlisted to RAC to Confirmed
metakgp visualize	visualize the contributions on Metakgp and see what user is making maximum number of edits! There’s also a CLI Companion for this. (I wonder why I thought it necessary to build a web page and a CLI)
Github repo creation calendar	In a particularly heavy repository creation period, I decided that I want to use the Github API to find out how often I create repositories. NBD.
Install tmux 2.0 from source	There aren’t any official sources and I didn’t want to have to figure out the dependency tree again and again
Understanding the TLS Handshake	Probably one of the most useful repositories I have ever spent time on. I learnt an incredible amount reading the TLS 1.2 RFC and it actually came in handy this December! So, the time I spent on this repository was certainly the most productive 4 hours of my life!
Youtube Chronological	Youtube will let you show the uploads of a channel sorted by date but you can only play them sorted reverse chronologically. When you have just found a [new channel][22], almost always, you want to play the videos chronologically. I tried to look for services online that would take a playlist and reverse them, but I couldn’t find any. So, I delved into the YouTube API. This Node.js script makes raw calls to the YouTube API. No API helper or anything.

It has been a long while since I have seen the repository pages of some of those utilities. It feels great to list out all the things that I built. Terribly great motivation.

Finally, I would love to see what constitutes your digital studio. So, if you have written a post like this before, do drop a link to that in the comments below. If you haven’t written anything like this yet, please, take an hour. Go through the list of your Github repositories and find the things that you built in the past and reminisce.

P.S. I love writing blog posts. double heart

Technological Unemployment - Summary of a summary

2018-02-22T00:00:00+00:00

There is this approach to consuming information online. I don’t know what it is called or who came up with it. The approach is to look at a topic, write down the solid questions you would like to answer about the topic, consume the material about this topic that’s in front of you and after you are done, answer the questions that you had started out with. With several blogs (esp Slate Star Codex), this approach has helped me a lot. Especially because it gives me an idea about what I would like to know, what my presumptions / beliefs about the questions are before I read the article and am influenced by the author’s presumptions and beliefs.

Technological Unemployment is a post on Slate Star Codex. I have been following this blog, albeit without understanding much on several occasions, for the past month and I jumped right out of my seat when I realized what this post was about. I had recently had a discussion about this with my parents in the context of the (lately increasing) AI paranoia articles in mainstream online and offline media and on sources like LinkedIn where people seem to be posting stuff about this all the time. (Elon Musk’s rather arbitrary position on this doesn’t help)

Before I read the post, I wrote down some questions and after reading the post (and taking notes, yeah.), I answered them.

Premise:

Prime Age Male Labor Force Participation Rate went from 97% in 1950 to 88% in \2014. Where did the ~10% of men go? Why aren’t they part of the workforce anymore? Were advances in technology the cause for their exit?

What sectors and income brackets were these 10% men a part of?

Caveats

This is third-hand information. Scott Alexander (the blog post’s author) read several papers and reports about the topic and composed his post. I have read his post and skimmed through the text and analyzed the graphs in the Brookings Institute Report.
My questions will not be the same as the questions you would like to answer depending on your age, field of employment, childhood etc.
The studies concentrate on the labor market in the US. So, it’s inherent in the blog post that everything is in the context of the American labor market. There will be wild differences between these numbers and the numbers for India or China, but the comparisons between the US and countries like Japan, Germany, Spain and other EU countries seems to indicate that the behaviour in these countries is more or less similar and the US labor market may be used as a proxy for these countries’ labor market.
The numbers in the tables have been taken from graphs that are mentioned below the tables. The numbers are representative and as accurate as can possibly be taken from a graph which doesn’t have labels.

That said, here goes.

Goal: Are people losing jobs because of advances in technology?

What age-group, level of education and gender dominates this set?

Throughout the analysis, we concentrate on Prime Age Male Labor Force Participation Rates. i.e Men between the ages of 25 and 55 who have a job. Female LFPR are not considered here because the main jump for Female LFPR came around 1990-2000 and it seems rather premature to make anything of numbers that are only 20 years old.

Age Group - All age groups; But mainly advanced ages; where disability can be a major cause for retirement before the age of 55 or a change of role at home (stay-at-home dad); even apart from this, there seems to be some indication that younger people have a degree of robustness in the selection of their field of work and an option to go back to college and gain skills as compared to older people.

Level of education - People whose level of education is less than high-school are the worst affected. Men with college degrees show a rate of decline that has remained more-or-less unchanged since 1950. The following table shows the Prime Age Male Labor Force Participation Rate distribution for three years across level of education:

Level of education	1968	1988	2016
4 years of college and more	97%	95%	94%
Some college but no 4-year degree	96%	95%	87%
High school degree, no colege	95%	94%	84%
Less than high school	94%	85%	80%

Figure 4, Brookings Institute Report

Gender: Inconlusive

What kind of technology lead to the loss of jobs?

It’s better to rephrase this question as “Which sector lost the most amount of jobs because of advances in technology?”

Manufacturing and middle-skill level, middle-wage jobs were the worst affected. The conclusion on this is unanimous and there are no doubts about who was dealt the worst hand here.

Manufacturing jobs as percent of total:

1948 - 30% 2010 - 10%

Manufacturing jobs (absolute numbers):

Year	Millions of jobs held in this sector
1970	18
1980	19.4
1990	17.9
2000	17.1
2010	11.5
2016	12.1

Figure 6, Brookings Institute Report

It’s worth noting that the share of manufacturing the in the GDP has remained fairly unchanged over this period of time.

Manufacturing has been losing jobs ever since World War 2. During the Industrial Revolution, there was an exodus of people from agriculture to manufacturing. Now, if there is to be an exodus, where would the people with the current manufacturing jobs go?

Surprisingly, it’s not the least-skilled jobs which lost out. In fact, they were the ones that gained because it’s fairly hard to automate these jobs away. These jobs are manually intensive or require some level of interpersonal communication and robustness (say against vandals or robberies in the case of truck drivers) which are not easy to solve in the real world.

A quote from the article that sums it up:

This is potentially consistent with a story where the jobs that have been easiest to automate are middle-class-ish. Some jobs require extremely basic human talents that machines can’t yet match – like a delivery person’s ability to climb stairs. Others require extremely arcane human talents likewise beyond machine abilities – like a scientist discovering new theories of physics. The stuff in between – proofreading, translating, records-keeping, metalworking, truck driving, welding – is more in danger.

There is one major caveat here though: As the number of jobs has always been declining, it’s hard to say without a shred of doubt that the jobs that were lost were indeed lost because of automation and not because people had a change of mind and decided to pursue other jobs or manufacturing jobs became less desired or some reason not related to automation at all.

Jobs were lost mainly in the manufacturing sector and in sectors that require medium skill level and are middle-wage

Was this technology cheaper over a period of time, compared to hiring people?

I couldn’t answer this question. Common sense seems to suggest that this should be the case or else why would businesses do something that costs more and employs fewer people? A quote from the blog post hints towards the same:

There may be some point at which we too stop being worth more than it costs to replace us. And the decline of manufacturing, the increase in labor force nonparticipation and despair in rural Rust Belt communities, etc, suggest that point is fast arriving.

This comment takes a stab at why even if one thing became cheaper, the inventors would be forced to move on to some other sector and if that continues, everything will become really really cheap but everyone will also be out of a job. Where did I end up?

Anyway, Couldn’t answer.

Did the jobs which were lost involve a hazardous work environment?

Jobs like those on the 24 hour assembly line that cause a lot of employee discomfort; toxic factories like the ones for fireworks, textile; works inside coal mines. In general, these are environments that adversely affect the employee just from working there. Dry cleaning is another example.

There’s no particular indication here that jobs in hazardous environment were more automated compared to non-hazardous environments. The Brookings Institute report does provide a good out for us though.

Table 1 reports the “Most Rapidly Declining Industries - by employment projections” as Apparel, Leather and allied manufacturing; Tobacco manufacturing; Postal service; Federal enterprises except Postal services; Manufacturing and reproducing magnetic and optical media. The first two sectors in this list are hazardous when the proper employee safety norms aren’t folloed, but that probably won’t be the case in the developed economies.

So, Not very conclusive; But some of the jobs which were lost did involve hazardous environments

Were the jobs that were lost monotonous?

The answer to this is a resounding yes. Just from the definition of middle-skill level, middle-wage jobs which have been the worst affected, this should be apparent:

Middle-paying occupations are stationary plant and related operators; metal, machinery and related trade work; drivers and mobile plant operators; office clerks; precision, handicraft, craft printing, and related trade workers; extraction and building trades workers; customer service clerks; machine operators and assemblers; and other craft and related trade workers

(The occupations in bold are monotonous, in my opinion. I am probably wrong, so please make your own judgement)

So, A resounding yes

Did the people who lost their job find another job?

If yes, was the new job higher or lower paying than the old one?

Once again, we are stuck without any data. But we have some strong indications about the trend here. The consensus seems to be that most of the workers who were in middle-skill, middle-income jobs who lost their jobs, slid into low-skill, low-wage jobs, because they couldn’t complete the requirements and acquire the skills to move to the high-skill, higher paying jobs.

This being the norm, there are certainly some people who do gain these skills and move into higher paying jobs.

Strong indications that most people who lost middle-income jobs, slipped into low-wage jobs. While the others either exit the workforce or gained the skills required to move to a high-skill, high-wage job

I conclude exactly as Scott Alexander does at the end of his post. I don’t think the people who aren’t part of the workforce anymore are not here because of the advancements in technology.

Technology has always been progressing forward and there doesn’t seem to be anything historical happening right now (and there aren’t any indications that something of the sort will happen in the next few decades)

Perhaps the rosy picture painted by more people leaving the workforce early to retire, take care of family at home, or because of disability is one worth considering; even if it seems too good to be true and can’t be accepted whole heartedly.

That’s the end of my third hand information dissemination effort. As mentioned earlier, definitely read the SlateStarCodex post. If you have the time, then read the Brookings Insitute report. It presents the data they have very well.

2 weeks of Monochromacy

2018-01-28T00:00:00+00:00

It all started with a tweet.

My phone is now two years old and I was looking for the next thing out there because I am bored with my phone and I really don’t want to buy a new phone. I am pretty sure this is the first time in my life I have actually not wanted to buy new hardware, even if I have the chance.

I like my Nexus 5X, but I had to do something to keep it interesting. Enter, Monochromacy.

Full disclosure: I have had ~150 tabs open on my phone browser for as long as I can remember. I never found the time to either read everything or send them to Pocket and forget about it for a year. I was reading a little bit too much news lately. My goal with this change was two-fold:

Do something with my phone that would keep it interesting to me
Spend less time reading news articles and keeping up with “current affairs”. Spend more time reading books, instead

2 weeks on, I think I have achieved both these aims and I am ready to continue with this experiment for a little bit longer.

The main question that comes to your mind if you haven’t done this before is probably “How drastic of a change is Monochromacy really?” Not a lot.

Almost nothing in today’s vanilla Android feature set depends on color. You can still send messages, read emails, visit web pages, scroll through the infinite time sink that calls itself Instagram, play a game like MMM Fingers. You can still take color pictures (they will appear black and white on your phone, but they are actually color. this is definitely not a doubt I had when I took my first picture after turning on monochromacy).

The two things that became unusable were (a) Read receipts on apps like Whatsapp and Signal and (b) several of Instagram’s features. Read receipts are self-explanatory, the double tick simply changes color and this is something that won’t be detectable at a glance, you can still go into message details or something and check this.

Instagram has several features which become unusable without color. It has a too-good-to-be-true editing screen in the Story section. The annotation capabilities; drawing, adding text, backgrounds etc. All the features are very powerful, but can only be used in Black and White (you can’t use any other color as these might have low contrast and not be visible on color screens. You must stick to Black and White). Filters also become pretty cumbersome to use because it’s not easy to predict how exactly the photo is going to look.

All in all, it has been a productive change. I have been reading more books (maybe because they are more interesting?) and I have been scrolling less through time sink applications (because I have grown out of it? Stop kidding yourself, human) I intend to stick with this.

Pragmatic GoLang Workshop - Follow-up

2018-01-22T00:00:00+00:00

As part of the Kharagpur Open Source Summit 2018, there was a Pragmatic GoLang workshop on 20th January, for 3 hours. Kshitij organized and ran it. By all accounts, it was a great success! He did a really good job with the slides and with delivering the workshop! The workshop was originally scheduled for 3 hours but it ran for nearly 4 hours, and people sat patiently and programmed alongwith him.

I was part of the team who was there helping him out with the doubts that everyone who came to the session had. These are some of the things I learnt in the workshop and some notes about GoLang for the future.

GoLang requires the opening brace of a function to be on the same line as the closing parentheses of the function definition:
```
  // allowed
  func Test(a, b int) {
      ...
  }

  // not allowed
  func Test(a, b int)
  {
      ...
  }
```
I remember reading about this in the Go “How to Format your Code” section and thinking to myself that this is perfect because this is the style I swear by whenever I write JavaScript!
Go doesn’t have a float type. It’s mentioned pretty clearly in the Types RefSpec but I ended up expecting a similar construct to int. I should start using int64 instead of just int so that the code works on all architectures!
Go’s switch doesn’t suffer from fall-through, the thing that forms the material for several of the trickiest questions about C++. This is GOOD!
Arrays can’t be resized. Slices are references to arrays. Slices work a lot of like Vectors in C++, I now realize. Slices are a very convenient data structure.
Functions are first-class objects in Go. With lambdas, functions being first class objects, and closures being treated without a frown, Go achieves several of the things that I love about JavaScript! (Go is such a mixture of features from every paradigm of programming)
Channels can be closed. This enables iteration over the channel using the for-range structure!

All this stuff that I have learned about writing “Idiomatic Go” enables me to go back to year-in-twitter and leprechaun and do things the Go way.

Sidebar: I gave a talk titled Node.js, GoLang and Concurrency on 21st January! It went well (I think). The video should be up soon. I am working on an appropriate HTML page with the resources I would like to distribute with the talk. (slides, code, further reading)

2017 Review of Podcasts

2018-01-13T00:00:00+00:00

2017 was the year that I started listening to podcasts. It’s a new form of media and I am psyched to have taken to it. Podcasts seem to be getting more and more popular by the day, and I am really excited to see what happens to this format!

Vikrant Varma recently published his 2017 Review Of Books and this post is inspired by his post. I have been reading books and I started cataloguing what I read on Goodreads only a couple years ago. On the other hand, this is my first year of listening to podcasts, so I am aiming to build a much bigger collection of what I listen to as the years go by! (This list is pretty small!)

Everything in this list is great (podcasts are easy to drop if you aren’t quite interested in how the story goes!) Everything in bold is INCREDIBLE! Aaron Swartz puts it rather eloquently: (quote changed for podcasts)

Podcasts in bold are those that were so great my heart leaps at the chance to tell you about them even now. If you only have time to listen to some of this, listen to them!

Podcasts

Criminal

This is one podcast that I absolutely LOVE. It’s not a crime podcast. It’s a podcast about the people who get caught up in the world of crime. Many of the episodes are still lodged in my memory because of how unique the story was. These stories are not the ones that will come up in movies like Taken. These are the stories that are too insignificant for movies. Phoebe Judge’s narration is totally on point. It’s hard not to love her voice!
Serial Season 1

Thousands of words have been written about why Serial is the best podcast from last year. In 12 episodes, Sarah Koenig went through the whole story of a 1999 murder and tried to “play detective”. She explains everything and has put up everything she has on the website. We get an intimate look at the whole crime, the people around it, why did he lie, why did she change her story, and so on. I have spent hours obsessing over this podcast, until I realised that the answer is unknowable.
Why’d you Push That Button?

A light hearted podcast about why we use technology the way we do. The narration and topics are all upbeat. The experts that they bring in in the second half of each episode are worth listening to because they give a window into the thought processes that go behind the design of the most popular products we use today!
Fear and Loathing in the New Jerusalem

Part of the MartyrMade podcasts series

Over 22 hours, Darryl Cooper goes deep into the Israel-Palestine conflict. The story doesn’t start at 1967, or at 1948, or in 1929, or before WW1. The story starts in the 1880s. I still have 10 hours (2 episodes) to go, but till now, I have already learnt so much about the conflict. It’s not a biased version of events. Cooper works overtime to ensure that the statements he makes don’t appear biased even under the strictes scrutiny. He spends an hour on a single riot, to ensure that he paints the complete picture, to ensure that he doesn’t miss a crucial detail which ends up as the reason for our bias towards one side or the other. Who’s wrong? Who’s right? These decisions are complicated and Cooper lets you make them. He simply delivers the story.

This is certainly not for the hobbyist interested in learning about the conflict. This is for anyone who finds the conflict fascinating. To go from a persecuted community in the early 1900s, to asking for a state from the Arabs during WW1, Israel is now in a position to give a state to the Arabs. I couldn’t quite comprehend how exactly something so spectacular can even happen, that’s the reason I started listening to this podcast. This paragraph sums it up:

This is so like a Hollywood movie. There’s the peasants. There’s the Jews who come to displace them. The Jews are successful in displacing them. But then, it’s time for the peasant hero to come in and either reclaim the land or come to a settlement with the Jews, depending on what sort of a movie you are watching. In real life though, there are no heroes. How far is this conflict going to go? Is there some solution to it? Who will find that solution? This is not a case of good vs. evil. Things are not that simple.

(Not an exact quote. This is the gist of what he said.)

Episodes

The American Dream - Criminal

Absolutely mind-boggling. What if you walk up to a teller and hand them a note saying “Hand over all the money in the register right now, or I will shoot up this place”? I had to read several news articles to make sure that this mode of theft wasn’t just fictional!
Ex Libris - Criminal

About The Man who Loved Books Too Much
Bloodlines - Criminal

How hard is it to change the way people think about you? Turning a new leaf is never as easy as it is portrayed to be. This is the story of living with expectations you don’t want to meet anymore, because you are a different person.
Jolly Jane - Criminal

Chilling. Jane was a not-so-jolly-at-all private nurse. Will make your skin crawl.
Eight Years - Criminal

Cyber stalking isn’t just a joke. It’s a traumatic part of life for some people. It’s shocking that there aren’t laws around this. It isn’t even treated like real-life stalking.
Melinda and Judy - Criminal

Identity Theft in the 90s
In Plain Sight - Criminal

An escape from slavery that is no less exciting than a Homeland episode
I’m about to Save Your Life - Criminal

Our instincts tell us to be polite as much as we can: Don’t ruffle feathers, don’t raise your voice, don’t get into a fight. If you are mild mannered and someone starts exploiting you, what do you do? This is the story of a mild-mannered engineer in the 70s who went through it all and did so rather nonchalantly. I was restless as I heard more and more of this episode, I wanted something to happen, I wanted it to be a movie basically. I wanted someone to come along and help him and punch the bad guy in the face. That doesn’t happen. A rude reminder of our docileness, even in the face of evil.
He’s Neutral - Criminal

The hilarious story of a statue of Buddha put up to stop garbage dumping becoming a landmark in Oakland. Lesson: Don’t look for help with your problems. Just go out there and solve them!
Baby Talk - Allusionist

Why do we all sound like idiots when we talk to babies? Don’t be embarrassed, we’re helping them acquire language.

Also, pronouns are a messy part of language and it’s better to save babies from that atleast when they are really young!
Half Measures - 99% Invisible

The metric system vs. US Customary units. Satellites have been lost because of the unit systems; millions of dollars worth of problems; and still the US hangs on. It’s not like the US ain’t trying:

What’s the deal?
Interview with Ryan Dahl - Creator of Node.js

Ryan Dahl: “I think Node is not the best system to build a massive web server. I would use Go for that. And honestly, that’s the reason why I left Node. It was the realization that: oh, actually, this is not the best server-side system ever.”
Casey Neistat and Matt Hackett on Live Video’s Struggle for Interestingness

Hustle!
Mystery House - 99% Invisible

The story behind the Winchester mansion: the house where construction went on for years non-stop, where staircases lead to walls, and doors open to reveal 30 foot drops. It’s very interesting! There’s a song about it and a movie is coming soon.

Year in Review - 2017

2018-01-01T00:00:00+00:00

This has been a pretty good year for my blog, I think(??). Of course, the metrics that I use are rather subjective and might not bear even a semblance to the truth. In my head, 2017’s quick recap would definitely feature these things:

I started 100DaysOfWriting on the 18th of February. That was also my first post on the blog in 2017, and in almost a year!
SHA-1 was broken for the first time with some binary padding data, two PDFs had the same SHA-1 even though they were different!
I watched a little bit of Veep, but then gave up after the 4th season! Selena was just about to become president. I wish I could continue where I left off, unfortunately, it’s too late for that now.
My git patch was ejected (not merged into master), but the patch made it as far as pu. I had a really good time reading the code that powers so much of software today. It was fun, I will pick it up again sometime in the next year.
I updated from Ubuntu 14.04 LTS to 16.04 LTS After a nerve wracking few hours of guess work, I was able to get encrypted ext4 partitions to work! (+ auto-mounting on start-up
I watched and absolutely loved Big Little Lies S1. Season 2 is going to be absolutely epic!
Homeland’s 6th Season lived upto the mark and then some. Quinn’s resurrection is on the cards, I hope
Cloudfare faced Cloudbleed and the buffer overflow error reared it’s head once more! Thankfully, GoLang is here to solve all the problems. (Right?!)
I struggled with a weird bug on Ubuntu where my mouse would simply stop working now and again. I am still facing this bug, but it has reduced to a much more acceptable number of occurences. The modprobe fix works for now.
I started reading Austen! I read Emma, Sense and Sensibility and Persuasion. Her other three novels are on deck for next year.
Handmaid’s Tale (Atwood) took the spot of “The Dystopia I fear most”, displacing 1984.
Demonetisation was an eerie reminder of the power that the state wields in this country
The internet came together on r/place and did something to a canvas that boggled everyone’s mind and captured my imagination. Oh, I LOVE the Internet!
I re-watched about 10 seasons worth of Seinfeld. I am not exagerrating.
United Airlines dragged a passenger through the cabin and everyone was hysteric about it. Ended up reading a book written by a flight attendant and developing a lot of empathy
I taped my webcam, updated my RSA keys to 4096 bits and enabled ufw to close all ports, all the time. Go Privacy!
I found the Nautilus update irritating but didn’t switch and am now quite happy about it!
I watched Masaan, Lunchbox, Contagion, and Another Earth for the umpteenth time
I interned at Elanic for 2 months! I learned a lot and it helped me get Node.js!
WannaCry happened, MalwareTechBlog stopped it, then MalwareTechBlog got stuck in the US (and is still there!)
I stayed with Chrome and then, switched to Quantum two days after it released. Quantum is the best browser I have ever used in my life till now!
I had a tweet blow up (to some extent).
I was stranded in Koramangala for a couple hours and then we waded through a feet of water at around 1 am to get home!
I started listening to Podcasts and I haven’t been able to go back to my way of life before. They are only getting better and more popular, so that’s one of the good things that happened this year!
I watched the 19 episodes of My So Called Life! That show did not deserve to be cancelled right in the middle of a season
I watched 2 seasons of Downton Abbey and I am absolutely in love with Sybil’s character
I read parts the TLS 1.2 RFC and wrote an overview. Considering that this came in handy in an interview I had later this year, this has been a very worthwhile use of time. Much more so than I had anticipated it at the beginning
I spent Durga Pujo in Kolkata and saw first-hand how chaotic a city can be!
I raved about NPM’s abrupt change to their download-stats API which got some attention
I obsessed over Primer (2004) and managed to untangle most of the mysteries. This is going right up there on the shelf of my favourite movies of all time.
I watched Arrival and fell in love with time travel, aliens, Amy Adams, science and language. All over again!

And here we are. 1st January, 2018.

P.S This post ended up using the pronoun “I” a lot more than I expected it to. I didn’t know how to phrase what I wanted to say appropriately without doing that. The excessive usage does make me uncomfortable, but I guess it can’t be avoided. Casey Neistat did the same thing when he made a review video. His videos are SO GOOD.

What does "wrapping text to 80 columns" mean?

2017-12-07T00:00:00+00:00

When you write a line of text that is really long, you can do two things. You can enter a “linebreak” or a “newline” after every few characters. Or you can not do that and let the line be as long as it needs to be.

“Wrapping text at 80 columns” implies that you have configured your editor to insert a “newline” after you have typed 80 characters on a particular line. The cursor moves to the next line.

Sample unwrapped text: Unwrapped text

Sample wrapped text: Wrapped text

The difference between these two formats might not be obvious to some at first glance. The unwrapped sample will look different on different screens. If your screen is really wide, you will be stuck with each line having about 20 words. If your screen is really small (like a phone screen) then you will have a flowing passage of text.

Unwrapped text on a 16 inch PC Screen

Unwrapped text on a 16:9 phone in portrait mode

Unwrapped text on a 16:9 phone in landscape mode

The wrapped sample will look good on all computer screens because 80 characters is some sort of a lower threshold (Yes, there’s a reason that 70-80 is the preferred line length). I am not sure where this threshold comes from, but I don’t think the reason matters. Although, you should be aware that wrapped text doesn’t look like a flowing paragraph on phone screens. The newlines in between will cause some quirks. But these don’t matter because no one is reviewing C Code on Phone screens (except some Git maintainers on a holiday)

Wrapped text on a 16 inch PC Screen

Wrapped text on a 16:9 phone in Portrait mode

Wrapped text on a 16:9 phone in Landcape mode

Note that in the wrapped text, no matter what screen size you see this on, the newlines occur exactly at the same place. Larger the screen, the more whitespace you see on the right of the text and below the text.

Whereas on the unwrapped text, the newline could be anywhere depending on your screen size (it’s not technically a newline as the text doesnt have any newlines, it’s just a linebreak that’s inserted by the software rendering the text). Also, the larger the screen, the more whitespace below the text. There will be no whitespace on the left or right of the text because the text will expand to accomodate it.

Some other weird stuff: Vim has a mode which can wrap text for you while writing but still not enter a newline to actually make it wrapped text. I don’t know about this mode and don’t use it. You shouldn’t use it either unless your use-case calls for it.

Note: I had to explain what wrapping text meant to people on GitHub, and I couldn’t find anything on the internet for it. The Kharagpur Winter of Code 2017 started on 1st December. Since then, I have been getting patches from people who are extremely new to GitHub, Open Source, Programming, Computers etc. Often, while explaining concepts to them in GitHub review comments, I realised that many of the terms that I take for granted are murkily defined. It’s also hard to find “what” they are online, which increases the information assymetry and ends up wasting the time of everyone involved. That’s the reason, I wrote this post.

NPM changes downloads-stats API and informs the world 38 hours later

2017-10-08T00:00:00+00:00

There are two parts to this post. The first is about what happened, chronologically and the second part is about why I was so disappointed with this.

Note: All the times in this post which are not annotated are India Standard Time = GMT + 5.5

What happened?

Around 10 AM on 29th August, 2017, I was proof reading my resume and clicked on one of the npm-stat.com links that I have there. I reached that page and realised that the number shown there was drastically low compared to the number that I already knew was the real one.

I was surprised. The first thing that came to mind was that NPM had deleted the old data leading to the downloads in 2015 being shown as 0. I immediately checked other packages. I checked some really famous packages like express and users like @sindresorhus. All of their downloads were being shown as 0 in 2015.
I dug a bit more and reached the code-base for npm-stat.com and from the line on the website “All data comes directly from NPM”, I realised that the website probably worked using an API from NPM itself.

npm-stat.com is a webpack based frontend JS app. I have some experience with them and thus, I was quickly able to figure out where the API call was happening.

It was these 3 lines:
```
  function getDownloadsUrl(pkg, fromDate, toDate) {
     return '/downloads/range/' + dateToDayKey(fromDate) + ':' +
              dateToDayKey(toDate) + '/' + encodeURIComponent(pkg);
  }
```
The request is of the form: /downloads/range/2015-05-01:2016-11-01/terminal-wallet
Now that I had found out the downloads API itself, I immediately checked if the data for 2015 was there or not. I found that it was. This was a relief. I immediately downloaded JSONs of everything from 2015 to that day, but this was where I noticed the rate-limiting!

I requested data from 2015-05-01 to 2017-08-29 but it only returned data starting from 2016-02-29, that’s exactly 18 months before the end date. I had found the issue!
This was the first real milestone. Once I realised what was going on, I immediately reached out to people:

(a) Open a ticket with NPM support

(b) Tweet to @izs

(c) Open an issue on npm/registry. Here again, the complete issue was explained.

(d) Send an email to Paul Vorbach who’s the maintainer of npm-stat.com.
All this was going down at 11 am IST on Tuesday 29th August 2017. That’s 10:30 PM PDT which is the timezone NPM support seems to be operating in. So, I wasn’t expecting a response from them until the next day at the least. I got an official reply on the support ticket 17 hours after at 4:44 am on 30th August 2017.
After all this reaching out, (it was around noon already), I sat down to work a patch for npm-stat.com. It took me about a day. Around 4 pm the next day, on August 30, I opened npm-stat.com #47.

There was one round of back-and-forth with @pvorb before the PR was merged on September 7th, a week later.
On August 31st 1 AM IST, 38 hours after I first noticed the problem, (I have no way to know exactly when the API was changed), they announced the rate-limiting on their blog.

The documentation on npm/registry that they are talking about was added in two commits:

chore: document downloads api limits at 2:08 AM IST on August 31st

fix: jan 10 not jan 01 at 2:24 AM IST on August 31st.

I found this from git’s blame view. Clearly, their plan was all along to make the change and inform later.

Why do I even care?

It’s just a downloads API, CHILL MAN.

No one actually said that to me, but you might be wondering why this was such a big deal for me.

Mainly because I have download numbers on my CV and when someone clicks on the link and sees a number less than the number they just saw and no apparent explanation, it’s a bad look. There’s almost no explanation that can be given in less than a few minutes time. (Hence, the request to Paul Vorbach to put a note on his website)

Also because this was unprecedented! This sort of an API change (i.e. a change without any prior warning) leads to every tool that depends on the API malfunctioning! Their official stand is that they will keep doing it in the future as well, and they will only follow-up with documentation:

We reserve the right to further limit API usage without warning when we see a pattern of requests causing the API to be unusable for most callers. We’ll follow up with documentation in these cases. Our primary goal is to prevent API use from either deliberately or accidentally making the service unresponsive for other users.

Conclusion

I like NPM. Irrespective of that, if you want to work with Node.js then you are absolutely going to HAVE to use it. There are many things that could have been done better here: more warning before such API changes, a tweet an hour before the API change was deployed would have been notice enough. Or how about adding a field to the /downloads api response that informed the user that the data is not for the complete range that they requested because of a new API change.

They have a monthly newsletter, a Twitter account, a blog and they chose not to post about this anywhere. I can only guess that they were undergoing an attack and they had to limit the API to fight that. I can’t believe anyone would have a planned move to limit data and not inform their users about it prior to the change itself.

Explaining a change 38 hours after deploying the change is just not acceptable for an API that’s so prevalent.

TL; DR

NPM limited their /downloads API to return data only for the last 18 months if you request for a range more than 18 months. They informed the world about it 38 hours after I first noticed the change. This lead to npm-stat.com malfunctioning and showing drastically low numbers because of years of data was simply not there.

Music Update - August 2017

2017-10-07T00:00:00+00:00

Ah, so now, I can finally get to it! (I have been wanting to write this article since August 31, but something kept getting in the way. Also, writing it is sort of a hassle because I have to get the screenshots etc. So, maybe I was putting it away.) On with it!

Quickly: There’s a recommendations section at the end. It has 3 songs I think you should listen to. If you are not interested in my droning on and on about lyrics and throwbacks and percieved similarities, press END on your keyboard and check those three songs out!

This month was about two TV shows: the Atypical Season 1 soundtrack and a couple of songs that come up in Modern Family!

The first two tracks are from Modern Family. The first one’s really old and I hadn’t ever heard of it. It’s a beautiful story to listen to!

Paint It Black from the Westworld sound track made a comeback. It has a great piano beginning. But the bass drum + violin accompaniment captured my imagination a lot more! I have no clue what Ramin Djawadi does, but whatever he does, I wish he keeps doing it.

I rememeber he also did the one good thing about the Game of Thrones season finale: the soundtrack to that long scene in which Cersei burns buildings up, the king commits suicide etc. Come to think of it, that track also reminds me of this with the deep bass and the sustained note. There were vocals in that track (it’s called Finale). But the organ is still predominant. His style is unique!
I recommend the Atypical Season 1 Soundtrack. I have linked a Spotify playlist of the songs that stood out the most to me. Heads will Roll and Love Struck were stuck in my head for days (still are!).

White Lies was the one song I had already heard in this bunch! When it came on in the series, I remember the scene also. She’s walking down the steps and going to Nick’s bar, and then the song fades out as she says something. I was so excited to recognize a song from a TV show. It was like recognising Paint It, Black when it came on in Westworld.
Before I go to the albums I heard this month, a couple of the outliers here. Trains by Porcupine Tree and Pokerface by Lady Gaga (the latter is not in the screenshot). Trains just came back as a whim, I sort of wanted to listen to the prelude to that song.

Pokerface came to my mind because of HOW SIMILAR it sounds to Heads will Roll! The same aggresive female vocals, the similar drop. The resemblance was pleasant!
Album 1: Bookends by Simon and Garfunkel

I had a particularly good month in terms of the number of new albums I heard this month! Three in total, 1 I started last month but the remaining two were wildcards!

Bookends is a really old album and I am certain that I am the last to get on the Simon and Garfunkel bandwagon. They are SO GOOD! I watched some of their live performances as well, nothing stood out.

My favourites are quite obviously Track 10: Mrs. Robinson and Track 11: Hazy Shade of Winter! I am obsessed with Hazy Shade of Winter. It’s only 2:17, which sucks. There’s literally nothing else I can even think about. It’s a CLASSIC! The way I found it was also a really surprising and funny event. I heard the whole album mainly because I had played Mrs. Robinson from the album on Spotify and Spotify played the whole album.

Sadly, apart from 6, 10, 11 and 12, nothing else really stuck. I am not sure why, probably just because it’s just so old and it’s from another era. Something like that.
Album 2: Broken Machine by Nothing but Thieves

I started with their debut album last month. And THIS MONTH! I started listening to it with Sorry and Amsterdam. But as I heard all of it, and repeatedly heard it all, I realised that I couldn’t find one track that I wanted to skip when I was listening to this album! It’s insanely time-consuming, but also so satisfying!

This album is sort-of the reason that I have such disbelief at how good this month has been in terms of new music and the amount of GREAT tracks I found.

From this one, I have to choose Tracks 1-7, 9, 11-13. There’s no going around the fact that I have left out Particles and two acoustic renditions. I really don’t want to leave anything out. Particles is like If I Get High and different from the other songs in the album and that’s why it sounded out of place to me. That is the only reason that I have left it out.

4, 5, 6, 7 have such good lyrics. Especially, this particular passage from Soda:

I’m an exception, it’s hard to accept because I try to be happy, but then I forget. They tell me I need to chill man, it’s all in your head Maybe I am paranoid

…

I once had a thought, but don’t know where it went coz I have been living on Soda and cheap cigarettes Maybe when I was a kid, I was dropped on my head, yeah, that would make some sense!

I was on a 3-hour journey from Kharagpur to Calcutta when these lines first registered in my brain. I then selected “Repeat 1” in Play Music and heard the lyrics again and again and then the song clicked for me!

This was a great throwback to the Petrol Bunk conversation that Aaron and Abe have in Primer.

Aaron: Maybe your life is in disarray and you start to wonder, what caused this? How did things get like this? … As if, if their mother had breastfed them, their lives would have been different.

Abe: Bunch of whiners

Aaron: Yeah, they’re a bunch of whiners. … What’s worse? Thinking you are being paranoid or knowing you should be?

(P.S. If you thought my Primer obsession had run it’s course, you’ve got another thing coming)
Album 3: Modern Vampires of the City by Vampire Weekend

I already said, this was one hell of a good month of music! Almost no repetition except for music I found this month and I liked most of what I heard.

I have already talked about this a bit in the last music update,

I found this album on Spotify Discover, which is KA-RAZY! My favourites are still tracks 3, 4, 6, 8 and 10! This album is the one I am going to go back to when I need something to brighten my day. The happiness in this album is contagious. A sample passage from Hannah Hunt, if you tend to not beleive me:

A man of faith said, “Hidden eyes could see what I was thinking”, I just smiled and told him, that was only true of Hannah, and we glided on through Waverley and Lincoln

Days were long and nights no longer, count the seconds, watching hours, though we live on the US Dollar, you and me, got our sense of time!

Common! What did I say?

Recommendations

Song	Album	Artist
Finger Back	Modern Vampires of the City	Vampire Weekend
Reset Me	Broken Machine	Nothing but Thieves
Hazy Shade of Winter	Bookends	Simon and Garfunkel

About this section: I know everyone has songs that they want you to hear. I also know how hard it is to listen to something and be present and not just glide through it. So, I am starting this section on music updates where I get to pick 3 songs from that month which I absolutely want you to hear. (I figure this will give the posts a sense of structure that they miss right now!)

I also plan to select from the less famous songs that I loved from that month, because I figure that you will eventually hear the popular ones. Like Sorry and Amsterdam from Broken Machine are blowing up on YouTube and you are probably going to hear it eventually!

A response to The Coming Software Apocalypse (an article on The Atlantic)

2017-10-02T00:00:00+00:00

This is a response to The Coming Software Apocalypse, an article on The Atlantic published on 26th September, 2017.

The approach that the article suggests for programming is Model-based design. I would like to cover some of my own past experiences with bugs that existed in the code I wrote and how the lack of confidence in the foolproof-ness of a piece of software leads to hours spent in thorough (not exhaustive) testing.

1. `undefined` leads to 15,000 emails

During the past summer at Elanic, there was a typical problem in the ExpressJS based backend that powered the primary REST API that runs the Elanic mobile applications. There were three ways for a user to sign-up:

a. Using their Facebook account b. Using their Google account c. Using an email ID and a password

(c) was majorly used on the iOS application where OAuth2 logins weren’t yet implemented. The issue was that when a user signed up using an email ID and a password, the email ID wasn’t verified. i.e. people could sign up with fake email IDs like fake@example.com and get away with it.

Solving this is pretty straight forward. When a user signs up, check if they are signing up with type (c) and then, send them an email with a link that consists of a verification token. When they click on the link, we will get the GET request and using the token we can mark their account as verified.

If the email ID doesn’t exist or is not controlled by them, then they will never be able to verify their email. EEZY PEEZY!

I thought about it a bit more and realised that there is one more case in which we would want to verify the user’s email and that is in when the user updates their profile and changes the email from there. In that case as well, we would have to verify the new email ID. (And if required, send an email to the old email ID informing them that the email ID has been changed. This wasn’t required.)

So, now there are two places we want to send the verification email: (a) when the user signs up with an email (b) when the user updates their email.

Now, the way the code was structured, the UserProfile model had a function UserProfile.updateProfile({ filter, update, options }, callback) and that was the right place for this code to be. Primarily because updateProfile was being called from across the application, and if in the future a flow is introduced where the email might change from somewhere else, this would take care of it.

So, I wrote something like this.

The routes looked like this (server.js)

// POST /users
// Creating a new user
function PostUsers(request, response, next) {
  let params = request.getParams;
  let profile = new UserProfile();
  profile.email = params.email;
  profile.password = hash(params.password);
  profile.save(error => {
    if (!error) {
      sendVerificationEmail({ email: profile.email });
      response.send(201);
      return next();
    }
  });
}

// PUT /users/:id
// Edit a user's profile
function PutUsers(request, response, next) {
  let params = request.getParams;
  let userId = params.id;

  let filter = { _id: userId };
  let update = _.delete(params, "id");

  UserProfile.updateProfile({ filter, update }, (...) => {
    ...
    return next();
  });

And the model function updateProfile:

UserProfile.updateProfile = ({ filter, update, options }, callback) => {
  if (update.email !== undefined) {
    sendVerificationEmail({ email: update.email }, () => {});
  }

  mongo.update(filter, update, (err) => {
    if (!err) {
      return callback();
    }
  });
});

Since I have spoiled the problems in the code for you, you probably found the problem?

It’s the first line of UserProfile.updateProfile! I check if the update.email is undefined or not. While I was writing this, I was convinced that this will check if the email has changed, because there would be no point to send the unchanged email alongwith a PUT call right? Why waste bytes on the wire for things that haven’t changed?

Well, I got hit right in the face! Turns out BOTH the Android and the iOS application send email alongwith the PUT call and let the backend deal with the change (or not). Worse still, even PUT calls from an internal tool called the “administrator panel” also sent this parameter! And internally, profiles are edited a lot to change tags, etc.

During the next 4 days, there were nearly 15,000 emails sent out of the Mail exchange. Several frequent users who had their profiles being edited from the internal tool received the “verify your email” mail multiple times, as high as 20 even.

This was one of the simplest features to implement, but there was still some nuance! THAT is the reason this experience stuck with me, even 3 months later.

2. My ongoing struggle with Merge Sort

I have been implementing sorting and those kind of things for quite a while now. I keep re-implementing them just to keep up-to-date and warmed up in my primary langauge for competitive programming: C (although I write it in a .cpp file and compile it using g++, more on that later)

Every time I implement Merge sort, I almost always start with the function signature int * merge(int *arr, int n) whereas the actual signature should be int * merge(int *arr, int begin, int end)

The first signature can still be worked, but I like the second one for it’s verbosity.

Then, you find the middle as middle = (end + begin) / 2 and call merge on the first and the second halves.

Anyway, there’s no particular bug here, I am just stressing on how although I know this algorithm inside-out, I still need to write it and test it with a couple of arrays before I can integrate it into my larger file. (The few times I was over confident about it and directly started building atop the merge sort I wrote and thought worked, I spent a significant amount of time going back and forth and finally realising that I had flaked out in the first function I wrote!)

My final code has been buggy in fairly limited instances. Through the development process, of course, there are several basic mistakes I make and discover and fix and move on. The iterative process that is so common is the one I subscribe to as well.

A couple of points about the article,

Newcombe isn’t so sure that it’s the programmer who is to blame. “I’ve heard from Leslie that he thinks programmers are afraid of math. I’ve found that programmers aren’t aware—or don’t believe—that math can help them handle complexity. Complexity is the biggest challenge for programmers.”

I am not sure what programmer is “afraid” of math. Anyone who comes from an engineering background or any core science background is almost definitely not “afraid” of math. If he particularly means Finite Automata and FSMs (which they talk about in the article as being the holy grail for programming, sort of. Read the article for the full picture) then even that is incredibly common. Anyone who has taken an engineering course with an interest in Programming (even those who don’t have a major in Computer Science) will definitely do a course on Switching and Finite Automata or Formal Language and Automata or Finite State Machines themselves!

There was one thing in the article though, which I think appears fiercely to everyone who writes code for a living:

Programmers, as a species, are relentlessly pragmatic

Samsung SM-B313E/D - Flashing software and getting out of a fix

2017-09-26T00:00:00+00:00

It all started with this tweet:

That’s a Samsung SM-B313E/D. I forgot the password and it was actually really disturbing because I couldn’t depend on myself to remember the several other passwords when I forgot a 4-digit password for an old phone.

That aside, I needed a way to fix this. There are several tutorials to do this on the Internet. A selection:

There was something peculiar about each of these ones. The phone that I am talking about is a really cheap phone (~ Rs. 2000 or USD 40) and thus there were no “high quality” tutorials for this, as one might expect for a smart phone like the Sony Xperia L or a Nexus.

All the tutorials showed more or less the same tutorial, but I was still not completely sure that I would be able to recover my phone using the tutorial. Also, being pretty paranoid myself, I saw no reason why they would not just ask you to run some EXE file on your Windows computer with administrator privileges and hide a virus inside that.

I decided to take the plunge, mainly because I had run out of options. Samsung Service Centers are not the best places to go to in the world if you want quick resolution.

The process was simple.

Install the Spreadtrum Driver Package
Download a .pac file which has the software packet for this phone
Use a software to flash this PAC file onto the phone (I found an XDA thread about this)

The verdict is that I was able to get the phone working once again! It works fine now, and there was really nothing off about the tutorial. I used Tutorial number 4 from the videos listed above. But for some reason the ResearchDownload tool linked in that video didn’t work for me. So I went to their website and downloaded the latest one.

Another weird thing was that the PAC File that I recieved said that the software was for SC6530_SAMSUNG: b310e, but it worked for B313E/D. This was one thing that I was most fearful about: that the wrong software would fry the phone. It didn’t! I guess the software packages are close enough replicas of each other to not make much of a difference.

There’s certainly a difference in the software packages. In the earlier software that was there on the phone, I could select multiple message and mark them as read. Now I can’t, I can only delete them. So, the software has definitely changed.

This was a success story. My heart was racing when the SPD Flash Tool was doing it’s thing, but it worked out fine!

Some notes:

Tutorial 4 from above
SPD Flash Tool’s website: http://spdflashtool.com/ (I think this is the official one, but I can’t really be sure. There are several mirrors of this software out there)
The PAC file and the drivers from Tutorial 4 above worked for me.

I used a Windows 8 computer.

I realized that I should probably put a disclaimer here, just to be safe.

Note: A typical disclaimer. I am copying text from the MIT License that absolves me of any responsibility in case of loss incurred from doing something that I state worked for me above.

THE TUTORIAL IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE TUTORIAL OR THE USE OR OTHER DEALINGS IN THE TUTORIAL.

Primer (2004) - A somewhat better summary with a sequence of events

2017-09-19T00:00:00+00:00

Spoilers, don’t read if you haven’t watched the movie

Since my first post about Primer, I have watched the movie a few times. I have read some of the recent (and even old) threads on r/primer. I think I understand everything a bit better now.

The First Revision from Abe’s perspective

Trying to draw a complete timeline from the beginning is not that useful, but turns out to be the easiest way to understand the details of the movie:

Sunday late night: Abe1 turns on the failsafe box with a few hourse delay and then goes to sleep.
Monday 8:30 am: Abe1 turns the box on, proceeds to the hotel and stays there (he remains unreachable via his work or cell number, as Aaron1 says)
Monday 8:50: Abe2 emerges from the box, there will be 2 Abes for the next 6 hours.
Monday ~11 am: Abe2 goes to Aaron1 and tells him about the machine. He shows him the Protein buildup thing, in Aaron1’s office. Aaron1’s colleague tells him that he should tell Will that the Ariel meeting is pushed to Wednesday. Aaron1 goes to talk to Will while Abe2 gets the car round front. Abe2 ends up inviting Will to Robert’s birthday party. Will is also going to bring his cousin along. This cousin is Rachel’s ex-boyfriend.

They now proceed to the lab where they see the protein buildup people. They go back to the “shop” and put their watches in, they realise what it is. Aaron plays basketball, he says they can publish. Abe2 and Aaron1 then get something to eat / drink and stop their car in front of the storage space place near freeway. It’s now approximately 3:15 pm
Monday 3:15 pm: Abe1 walks into the storage space alongwith his Oxygen tank, while Abe2 and Aaron1 watch. “Who’s that, Abe?”, Aaron asks.
Monday ~3:25 pm: Aaron1 and Abe2 go to the room and see the machine. The machine is turned off. Abe1 is going back in time inside the box. They don’t disturb anything, and leave after some time.
Monday ~4:00 pm: Aaron1 and Abe2 are sitting in the car next to Aaron1’s truck. Abe2 gets a phone call from Rachel, he probably tells her that he’s going to be at Robert’s birthday party during this conversation. This is the reason Rachel appears at the birthday party. Abe2 and Aaron1 are washing dishes at home, they do NOT go to the party.
Monday (before they return home for dinner and dishes): Abe2 and Aaron1 sit near the airport and discuss what Abe1 did. Presumably, they also build the box that Aaron is going to use. (1)
Tuesday 8:30 am: Abe2 and Aaron1 turn the box on, spend 6 hours at the hotel. They use Abe2’s car to get to the hotel.
Tuesday 8:49 am: Abe3 and Aaron2 emerge from the box! They take Aaron’s truck to go to Abe’s house.
Tuesday before 12:30 pm: Abe3 and Aaron2 buy stocks of RGWU, the price is going to double in afternoon trading. They are going to make a lot of money.
Tuesday 3:15 pm: Abe2 and Aaron1 turn the box off, about 5 minutes later, they get into the box and start travelling back in time.
Tuesday 3:15 pm: Abe3 and Aaron2 are watching their doubles Abe2 and Aaron1 walk into the storage facility. The machines are running now, but their doubles will shortly turn them off.
Tuesday evening: Abe3 and Aaron2 discuss with Kara about what they would do if they had a lot of money etc. This is the point where Aaron2 puts the idea out there: “Punch Platts and tell myself not to”

Notably, probably tongue in cheek, Kara’s dialogue: “My husband, the hero” (knowing what’s going to happen next, this might very well have been the seed that planted in Aaron2’s mind that he can actually go back and change the party, although I wonder if Aaron2 (a) knows about the failsafe machine that would let him go back that far in time (b) knows about the shenanigans that Rachel’s ex-boyfriend pulls at Robert’s party. The whole day they have more or less not spoken to anyone, so I am skeptical on this.

More importantly, Kara asked Aaron (no number, because to Kara there’s only one Aaron) to call pest control to clean the Attic out. But Aaron2 says that they are only birds and doesn’t call them. Is this because he really believes that they are birds, or because he knows that there’s an Aaron in the attic? (clarity on this later!)

(When does the grocery shopping scene and the petrol bunk scene happen? Tuesday evening, Wednesday morning?)

Wednesday morning, 8:30 am: Abe3 and Aaron2 are going to take the trip again. They talk about how Aaron could tell Kara about it. The machines have been set to turn on at 8:45 am. Interestingly, one thing they do differently this time is that they take Aaron’s truck to the hotel instead of Abe’s car. Probably because Aaron wants to fill it up with gas? That seems to fit.

So, austensibly, the petrol bunk scene happens on Wednesday morning. (?)
Wednesday morning, 8:49 am: Abe4 and Aaron3 emerge from the box. Aaron3’s ear is bleeding, Abe4’s hand is. They talk about the dream and the tides that both of them seem to hear inside the box.

Notably, this time, Abe4 and Aaron3 have taken Abe’s car instead of Aaron’s truck to return to Abe’s house.
Wednesday afternoon: Abe3 and Aaron2 are at the library, still talking about telling Kara, flying to Costa Rica, etc. They decide to not tell anyone though. Aaron2 also mentions the innocuous “build a bigger box, because this one feels like a coffin” The word coffin will be re-used and is important.
Wednesday 3:15 pm: Abe3 and Aaron2 get into the box once again.

And now, Aaron has become the hero

Wednesday late night: Abe4 goes to Aaron3’s house and tells him about the Platts plan. They come out, see Granger, follow him around. Granger is now in a coma.
Thursday early morning: 3 days and 22 hourse after the failsafe (not the original one, as Abe expects it to be, because Aaron has used it in between) was turned on, Abe4 gets into it alongwith the 3 oxygen tanks, water and the triazolam for his next 4 days inside the box. He is going to use the reset button, go back in time and not tell Aaron about any of this.

Till now, things were linear. Suddenly, there’s the scene where Aaron went to Robert’s birthday party. From Aaron’s dialogue later in the movie, we know that there were two versions of the party:

The one which Aaron didn’t go to
The one which Aaron went to and rushed Will (the ex-boyfriend)

(1) happened only once. (2) happens again and again, as the narrator illustrates. Aaron re-does it until he gets everything perfect, exactly as he wanted it.

In the timeline when Aaron does go to the party, which is what the world will remember because the final revision does have Aaron going to the party. In that revision, Aaron goes to the party on Monday night. Abe may or may not go with him.

The Second Revision from Abe’s perspective

Sunday late night: Abe1 turns on the failsafe box with a few hours delay. He then goes back home, and sleeps.
Sunday late night: Abe4 emerges from the box. He goes to his own house and eliminates himself because that is the only way to take control of the timeline.
Monday morning: The box remains unused, Abe has probably decided to not use the box because he has seen the consequences and is definitely unnerved by them. He believes that Aaron doesn’t know about the box yet because he hasn’t yet told him about it.
Monday afternoon: Abe goes to talk to Aaron, probably planning to tell him that he was unable to understand the “box” and they should move on with some other project because they have clearly hit a wall with this one. Instead, He faints because of his long, 3 day, no-food-just-water hiatus inside the box. Aaron-n, meanwhile, has a recording of the day’s conversations and is going to try and redo everything as it is to reverse engineer the party moment and be the hero once again.

Aaron knows about the box whereas Abe believes that he doesn’t. Now, the HOW section will start making sense.

In this revision, Abe knows about the box. Aaron knows about the box as well. This is the atmost information symmetry that they have ever had.

Abe knows it’s consequences, having gone till Thursday. Aaron-n doesn’t, he hasn’t gone till Thursday early morning when Abe uses the failsafe. Nonetheless, he knows what happens till the night of the birthday party.

As the narrator says, “At this point, there would have been some discussion”

*1: I feel like they are sort of stretching it here. They discuss what he did, plan their next morning. Apart from they also have to build a completely new box for Aaron1 to use the next morning. They don’t go the party the first time, so this gave them something like 7-8 hours to do everything and get back home, have dinner with Kara and Lauren, do the dishes, and wake up next morning to execute the plan. Since they didn’t go to the party, and Abe was around to tell him exactly how to build the box, this might very well be possible. It just feels really fast.

The one big question I have now is How did Aaron find out about the failsafe? When did he use it?

When he did use it, he went to Aaron’s house right after emerging from the failsafe (Aaron in the Hoodie), he drugs the milk and puts Aaron in the attic. Now, he is in control. The first time Aaron goes back, he records the conversations of the day. The next time he goes back, he brings the recordings with him, and then follows the script.

He also brings back a coffin with him at one point, because then, he can emerge from the box, start the coffin up and then go on with his Monday morning. That way, he will always be able to go back to Sunday morning. ( Remember that as soon as someone emerges from a box, they can’t turn it off. They MUST wait till atleast the time the machine has been turned off, their double has gotten in and it has cooled down before they can start it again. This is the meaning of one-time use only, they can use it only one time to go back to a particular point in time, the point when they start the machine. )

Finally, when Aaron comes back the first time, he drugs his double and puts him in the attic, but he can’t keep doing that. Starting from the second time, the Aaron who was already there convinces the Aaron who just emerged from the box to leave town so that he can continue on his plan. Seeing that the narrator left, it’s fair to assume that most of the Aarons would.

In my previous post, at the end of it, I ask a few questions.

Q1: Granger, how did he come back?

I have started to believe that Granger’s appearance is a jarring reminder to Abe and Aaron that they are NOT prescient because they might know the past, to the last point in a basketball game, but they have no control over their future selves who might do whatever they think is fit.

Q2: Fail-safe, who built this machine? Why?

Clearly, Abe built the first Failsafe. While building a machine, knowing what the machine was doing, he built two machines and turned one on before Monday. Probably Sunday late night. Now, he has the ability to go back to Sunday late night and not tell anyone about anything, if he so wishes by taking care of his double. He would have to kill him / incapicitate him, so as to take control of his character.

Although, by all accounts, Aaron used the machine before Abe had the chance to. This is the debt that Aaron owes Abe (I think), that the narrator alludes to “Now, I have repaid any debt I may have owed you”

Q3: What is the exact sequence of events?

I have tried to answer this for the first part of the movie and that part is very clear to me now. There are a new set of questions though:

How does Aaron find out about the failsafe? Why did he go to the storage facility and look at the manifest without Abe being with him? Did he go there so he could build his own failsafe and put it in another room, just as Abe did and happened to notice the two rooms registered to Abe?
When does Aaron use the failsafe? Does he use it on Tuesday night?

Primer (2004) - Watch 1, impressions and questions

2017-09-10T00:00:00+00:00

Well. This movie is one hell of a mind-bender.

MAJOR SPOILERS AHEAD

This is a great movie that you don’t want spoiled. Watch it yourself and you will have a GREAT experience trying to figure everything out. (Not that I am there yet, but I have a feeling it will be great!)

Of course, I didn’t understand everything. I sort-of have a sketch of what really happened, but I am not sure on the details, and the last 15 minutes of the movie totally went over my head. For this watch, I am just going to list the things that I could make out. This definitely simplifies the plot, but the specifics are still hard to get.

First of all, the characters.

This is Abe:

And Aaron:

Abe’s girlfriend is Rachel Granger, and her dad Thomas Granger is “their last, best hope of funding”.

Aaron’s wife is Kara. They have a daughter Lauren.

Philip and Robert are two people who appear in the first scene, and once after. They don’t really matter (as for this viewing), so they can be safely ignored. It’s only Abe and Aaron who matter.
They don’t like / hate Robert Platt. This is something related to the funding, it got pulled or something. The hate is important because Aaron, who is generally defending him, later proposes an idea where he can punch Platt once, but then stop himself from doing it. Thus, having experienced the punch but not having to live in a world where there would be consequences. More on this later.
Chronologically, there’s some science-y stuff and they build something. It’s not really clear what they are building. The mass of objects they keep inside decreases after some time. At a first run, the machine just sort of runs smoothly for a bit but when they run it again for the box, the machine vibrates a lot and then just stops.
Soon after the first run, Abe is asleep at his place, at 7 pm. Whether it’s already started, I can not tell. There are two people on his couch, Brad and the “Man on the Couch”. This again stumps me.
Anyway, Aaron shows Abe a mechanism with which he has a stable method to reduce the mass of something by 10%. Still quiet unclear about what’s really going on. They also notice that unplugging the batteries keeps the machine going for a few minutes before it shuts down. They call this “blocking gravity” inside the box.
Now, about 20 minutes into the film, is where the looping begins. We aren’t really sure who it is, but an Abe walks out into the terrace of an office building. The camera fades into the shot of Abe walking towards an Aaron. So, the walking Abe might not be the same as the terrace Abe, and actually, I think they aren’t the same. It is worth noting here (and I just realised this) that the first time this happened, Aaron was NOT able to reach Abe the whole day on his work or cell because Abe is already doing this every day! He’s already called in sick, and has left his cell phone making him unreachable, and maintaining the perfect symmetry each day. This happens somewhere around noon because after this, Abe takes Aaron to the protein thing, shows him all of that, then they go to the lab where they do the watch thing, and finally, when they have eaten, it’s 315 and Abe is walking into the storage shed to lie there for the next six hours to come out as a new Abe.
From the end of the movie, it’s clear that the Aaron sitting on the bench with the headphones, is listening to the conversation that’s about to happen. There had to have been a time when it happened for the first time, or the second. As the narrator says at around 64 minutes, the second time Aaron came back, “It wasn’t so easy. He didn’t expect me to put up a fight. But I realised he simply wanted it more. He had already performed the tasks I had intended to, of recording the conversations of the day, as I had intended to.” So, the second time Aaron came back was when he met the narrator, fought with him and took the narrator. Who is the narrator? (That’s a question I have been unable to answer, still)
The night of Robert’s birthday party is the same day as the time when Abe tells Aaron for the first time what the machine really does, the same day that they put the watch into the machine for the first time and figure out what it really does. The same day with the protein build-up sequence. Rachel goes to the birthday because Abe asks her to, and Rachel’s ex-boyfriend shows up with one of Aaron’s friends. The ex-boyfriend brings a shotgun out and Aaron rushes him, eventually saving everyone. They call him “the hero” for doing this. Abe reprimands him at having done this without thinking about Kara and Lauren. The first time that this happens in the movie, Aaron doesn’t have headphones plugged in, which leads me to believe that this reprimand from Abe is the first time that Aaron even does this, and Abe is NOT at the party this first time because he doesn’t know about it and has to hear about it from Robert. He decides to go to the party after he knows what’s going to happen there, Abe definitely does NOT go to the party the first time.

This particular event becomes an obsession with Aaron and Aaron seems to keep repeating this sequence, again and again until he is satisfied with how events turn out. As the narrator says, this might have taken 3 or 4 times or even more. No one can say. This sequence, and the constant changing of what’s happening seems to be what makes the two of them come undone from the routine they had going which made a lot of money but didn’t have any recognition from anyone.
The first time the protein buildup scene comes up, Aaron is wearing the headphones, which means it’s not the first time he’s going through it. He’s actually already gone through it once and is simply following the script to keep everything the same.
The scene in the movie where the put the watch in and notice the 1300 thing, doesn’t have Aaron with the headphones which perhaps means that that is the first time that that happened. Note that by this time, the human sized box already exists and is in the storage shed. Abe has already built it. Aaron still is only understanding the box, whereas Abe has gone ahead, built the box, gone back in time, and is NOW explaining the box to Aaron. (while Aaron’s equivalent Abe in this world is actually seen by Aaron walking into the storage shed at the time they walk in through the binoculars)
So, clearly Abe has been doing this more times than Aaron has. Abe also figures out that he can make money by simply figuring out which stock grew the most in the last 6 hours, then go back, get out of the box and buy that stock. Essentially, starting 830 am, Abe and Aaron spend 6 hours in the hotel, 6 hours in the machine and were able to go back in time, to eventually get out of the box at 850 am.
This is a small drawing I made of how the double really comes to be: (wherever there are two lines overlapping, the doubles co-exist)

That’s about all I have time for now. There are still several things that are unexplained:

Thomas Granger: What’s he doing? Why’s he following them? How did he travel back 2 or 3 days? Why would they even let him know about the machine or let him use it?
Fail-safe: Who built this machine? Why did he build it?
The killing in the failsafe machine’s room, Abe kills someone. Is it Abe killing himself or Abe killing Aaron? I am leaning towards the former.
What is the exact sequence of the way stuff goes down? (This is the question that will finally hold the answer!)

Clearly, this is a GREAT movie.

Auxilliary material for TLS 1.2 - TLS Parameters - Registries

2017-09-07T00:00:00+00:00

Part of the TLS explanation series: Auxilliary material

After understanding the TLS handshake, there are still many moving parts that have still not been satisfactorily explained. Perhaps the most important in those is the assignment of the byte values for each of the cipher suites, handshake message numbers, etc to the different things that have predefined lists.

These are defined in the TLS Parameters document, maintained by the Internet Assigned Numbers Authority (IANA). If you didn’t already know, the IANA is also responsible for allocating Autonomous Systems on the Internet backbone which is used for routing packets around the network. They are the ones who have cordoned off 10.0.0.0/8 and some other IPs for LAN usage.

Anyway, this document consists of several registries. I will try to cover some of the common ones that I understand:

Cipher Suite Registry

Perhaps one of the most frequently changed registries, this registry keeps getting new Cipher Suites added to it, and the corresponding byte value for it.

It also mentions the RFC where we can find the reference for it. The reference consists of a short description of the algorithm, and the associated Pseudo-Random Function (PRF) to be used. The PRF, as you might remember, is used to generate the keys for the actual Record layer message transfer.

A common cipher suite on the web is TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256. It is being used on this website by the Cloudfare SSL proxy. Breaking down this Cipher Suite:

Key exchange is done using ECDHE
Key exchange is authenticated using ECDSA
Bulk encryption will be done using AES_128_GCM which is authenticated inherently
PRF used will be the TLS PRF with SHA256 as the hash function

This last point comes from RFC 5289, section 3.2. So, for every cipher suite this whole thing is defined. Again, we see that the emphasis is less on making all the data available at the same place and more on (a) making sure there’s no implementation ambiguity and (b) making sure the system is massively extensible.

Handshake Type Registry

If you have seen actual session negotiations in Wireshark (or the transcript linked), you will know that each Handshake protocol message has 1 byte (= 8 bits) to indicate which Handshake message it is. This registry assigns those numbers, the ClientHello is fittingly 1 and Finished is 20.

Signature Algorithm Registry and Hash Algorithm Registry

These again are single byte values which have been defined for easy reference to each of these algorithms, without explicitly using strings etc. NEAT! (Packet sniffers like Wireshark take this into account and print the algorithm in the human readable form for convenience!)

Exporter Label Registry

This is yet another registry which has a very, very specific purpose. It stores the list of strings which can be used by protocols that wish to export TLS’s keying material to their own layers. This section neatly sums up the importance and the application of this registry.

These applications imply a need to be able to export keying material (later called Exported Keying Material or EKM) from TLS/DTLS to an application or protocol residing at an upper layer, and to securely agree on the upper-layer context where the keying material will be used.

Understanding and Explaining the TLS Handshake

2017-08-29T00:00:00+00:00

Yesterday, I open sourced tls-handshake, a web page that tries to provide an over view of the TLS 1.2 handshake at an abstraction level that is appropriate for students who have taken basic cryptography courses either at the college or school level. It has been one hell of a ride!

The idea came from my lack of understanding of the TLS 1.2 handshake, I really had very less of a clue about what was going on. I often checked certificates, especially on banking and merchant websites where I was entering sensitive credentials. This had become more of a habit. But I was only checking the trust chain of certificates, and the fingerprint and if the browser was giving me any warnings about it all. I didn’t really know what was really going on underneath everything, the math!

There was one obvious place to start: IETF RFC 5246: TLS 1.2

The spec is a dense 100-page document that goes over everything in implementation-level detail, i.e detail enough to ensure that any person who is implementing a TLS 1.2 client OR server faces absolutely no ambuiguity. The RFCs

go through a long process in which the TLS Working Group makes updates to them, their mailing list is a great place to hang out if you have a few hours on your hand and want to spend them learning more about how the next HTTPS spec gets written.

RFC 5246 is a great document, if you know a few basics that the document itself doesn’t touch on. These include some common cryptography concepts like asymmetric and symmetric cryptography, digital signature algorithms, the Diffie-Hellman Key exchange system. Apart from the mathematics of these which the RFC doesn’t attempt to explain or ellaborate over, the RFC is self sufficient.

A simple new session packet capture should be enough to tell you about the different set of messages.

This screenshot from the RFC itself, which I noticed quite late in my reading will tell you the sequence in which these messages are sent:

A few hours should be enough for you to understand the over-all flow of the information that is being exchanged and the different places that you might end up in, after each message is sent.

That is when I was ready to start writing the webpage. After beating around for a few minutes about a good UI to show everything, I decided to stick to text and simple HTML as much as I could, and use Markdown to format everything!

One thing I absolutely wanted to have was informal “Purpose” sentences for each message. Mainly because the TLS handshake is just a secure way for two people to talk, in a public place like a train or a bus station and without spelling it out, and share a message that they don’t want anyone else to know about. They have never met before. And both of them are pretty good at mathematics that involves huge primes, or elliptic curve points, whichever one is to your taste.

That’s sort of my story behind the “TLS Handshake” explanation page! I posted it on r/https (which is a pretty dormant subreddit), I might post it on Hackernews soon, once I can take some time off and go back to read the whole thing and be sure that there’s nothing wrong that’s written there.

Music update - July 2017

2017-08-12T00:00:00+00:00

The past month has been pretty good in terms of the amount of new music that I have heard (although the albums are pretty old themselves)

The Suburbs - Arcade Fire - 2010

This album won the Grammy Album of the Year award in 2010. Before this, I had heard only the second track, Ready to Start. That I absolutely loved, but after listening to the whole album, I am thoroughly spoiled now and I don’t know what to pick.

We Used to Wait, Sprawl I, Sprawl II: I am not inseparable from these three now. The two Sprawls tie seamlessly into each other. And We Used to Wait is such a strong throw-back to the time before smartphones were ubiquitous. (I was born in 1996, so I might not be that old, but I certainly remember the time before all smartphones had great cameras and everyone was posting photos all the time a.k.a the pre-Instagram pre-LTE world)

These few lines from We Used to Wait, I connected with rather too much.

(I made that with Gimp and the Solarized color scheme.)

City with No Children, Month of May, Modern Man, Rococo and Speaking in Tongues are also great songs! The more I listen to this album, the more I realise how good it is as a package, as a story.

Finally, a paragraph from Sprawl I:

Cops shone their lights
On the reflectors of our bikes,
Said, do you kids know what time it is?
Well sir, it's the **first time I've felt like something is mine**,
Like I have something to give.
The last defender of the sprawl.
Said, well where do you kids live?
Well sir, **if you only knew what the answer's worth**,
Been searching,
every corner,
of the Earth.

Sprawl I is the shortest full track in the album and it has barely three paragraphs of lyrics and yet, it manages to capture so much of the singer’s emotions in it.

(I totally saw all the Suburbs interviews that Arcade Fire did in 2010, especially with Regine. * _ * )

Nothing but Thieves - Nothing but Thieves - 2015

Nothing but Thieves is fairly new and they started only in 2012 and their only album was released in 2015. Recently, their new song Amsterdam has been really popular on YouTube.

In any case, this whole album is really bass-heavy and a few of the songs have a leading bass guitar track (especially, Wake Up Call and Honey Whiskey) and in every song the bass guitar is clearly audible, you don’t even have to strain, provided you are on a fairly bassy speaker system. (My little JBL In-Ear headphones haven’t really felt bass ever, so when listening on them, the bass feels significantly cut. Funnily enough, that’s how I realised how bass-heavy this album is!)

I can’t choose favourites yet because all of the songs are groovy and there’s no one song that I clearly don’t like. I specifically like Excuse Me, Itch, Honey Whiskey, Hostage, and Trip Switch. It’s a 16 track deluxe album and I have named 5 already.

If I Get High and Neon Brother are slightly away from the over-all tone of the album but if you don’t listen to them as an album and rather as singles, then they too fit very well!

P.S. I started writing this post on 6th August and I have finished it today. Since then, I have also heard Vampire Weekend’s Modern Vampires in the City. It’s a 2013 cheerful, indie-rock album. It’s REALLY good!

The Siege of Trencher's Farm vs. Straw Dogs (2011) - How loose an adaptation is it?

2017-07-23T00:00:00+00:00

I just completed The Seige of Trencher’s Farm by Gordon Williams. I also just completed watching 2011 remake of the loose adaptation Straw Dogs (2011).

Fair Warning: HUGE Spoilers ahead.

The adaptation is pretty loose, the whole sexual angle that Kate Bosworth brings into the movie is completely non-existent in the book. There’s also a child in the book, and Louise herself (George’s wife) doesn’t seem to have lived in Trencher’s when she was a kid. She’s from England but not exactly from the place that they have come to stay at for a year.

So, in the book, George, Louise and Karen (8) come to stay at Trencher’s Farm for a year.

Same in the movie, only it’s Amy and David Sumner. To add some more to this, Amy used to live in Blackwater and the farm is called the Wilcox Farm. Amy’s dad used to stay here and she had a thing going with Charlie Venner, the local hunk (played by the creepy-looking, tall Alexander Skarsgard).

The boys on the outside

The character names are quite similar. There’s the huge Tom Hedden (or Heddon), a local drunk who used to be a coach in his day. In the book he has a huge family that’s over-run with work and Janice Hedden is an afflicted daughter of his.

Norman Scutt is different though. Scutt is the one with the brains in the book. He makes all the crucial decisions during the siege itself. In the movies, he’s just a married man with a pregnant wife and he covets Amy.

Chris Cawsey is also different. In the book, Cawsey is some kind of a psychopath who likes killing animals and wants to get his knife into a human being. He is seriously mentally ill and needs help for sure, but in Dando, outsiders can’t come in to help. In the movie, he’s just a normal kid who’s hanging out and idolizes Charlie for he is.

Bert Voizey is a character in the books that I am particularly fond of. One of his introductory lines was incredibly funny and sad at the same time. It was this:

The third man - Bert Voizey - never felt comfortable in this kind of fancy house. Like one of his own ferrets, he had a natural instinct for creeping about in darker corners. He was not at ease with loud, confident people who stared you straight in the eye when they talked to you.

We later learn that Bert posions rats for a living and he’s proud of this because most people are freaked out by rats, but he understands them and does doesn’t have to be scared of them. Bert is missing from the movie.

Bic is a character who’s only in the movie. He’s a pretty 1D character and there’s not a lot to talk about with him.

Finally, about all these male characters who organise and are on the outside, there’s one summarising lines right before they start it:

All of them thought of different things, yet they were all there for the same reasons. They were the men nobody took notice of. They were the men who’d never had any luck. All their lives other people had told them what to do, had insulted them, put them in gaol, sneered at them, kept them poor.

While a lot of this might not really be true, this is what drives them.

The 2 Protagonists

With all of that talked about, we can finally come to the main two characters. George and Louise. David and Amy. It’s important to understand two things, according to me. I was pretty confused about their equation until almost the end of the book and the movie. It’s different in both. So, here’s a good place to branch out.

David / George

He’s an academic and wants to spend some time in a quiet, peaceful country side home so that he can complete a book (script) that he’s working on. Being an academic, he’s this ultra-modern husband character who’s always in control, who never gets angry. This particular trait of his irritates and infuriates his wife to no end. She wants to see some color in him.

He was so damned anxious to keep control of himself. He acted the role of a reasonable steady, dependable husband. In her general state of unreasonable resentment she saw this as an insult; if he was sincere he wouldn’t need to act, to keep such tight control of himself.

It’s also really hard for him to acknowledge the fact that there are some things that he simply can not do. Like when they find the cat’s corpse, he’s almost exactly as disgusted and scared by it as his 8 year old daughter Karen. Louise is the one who has to step up to take care of that particular problem. This kind of un-manly behaviour in an old-fashioned sense that he seems to lack really cuts him deep.

This really comes out when he’s completely realised his situation during the Siege. When he’s realised that he’s the man of the house and he HAS to take care of his wife, his child and the child-like Niles, because, as he puts it himself, They are my responsibility. Although no one tasked him with it, he feels the obligation to take control and ensure that everyone is safe.

One last thing about him that he himself seems to not have track of is the fact that his wife expects him to be an old-fashioned husband. A husband who beats her when she disobeys or does something that he disapproves of, as masochistic as it sounds, I don’t believe that Amy / Louise is masochistic. I think it’s the responsibility wearing of. George says that it was his wife who took care of everything when they moved to England. He wished he had done it, but doesn’t seem to resent it too much. As soon as they are there though, Louise is probably worn out from all the work that it must have taken to move a family of 3 from the US to England. And perhaps she had this belief that once they were there, George would take control and she could go back to being the person who followed rules instead of having to make them. THAT is the equation the couple share and THAT is the central question in the movie.

When someone is trying to get into YOUR house and hurt the people that YOU are responsible for, TO WHAT LENGTHS WOULD YOU GO TO PROTECT THEM? DO YOU HAVE IT IN YOU TO KILL, if that’s what it took?

It’s like all those childhood fantasy situations that kids often put themselves in and try to think about what they would do if they ever were there. eg: Man to man, a gun in each hand, would you talk or would you shoot first and hope to God your aim was right? eg: You are up against a guy who’s obviously physically stronger than you, would you try to outwit him or would you fight him anyhow, in your naive belief that the good people win?

It’s a delightful book to read, if you really have the stomach for it. There’s a lot of confusion and there’s a lot of pages that don’t tell you a lot about the story, but that’s simply because you gotta love George and Louise if you must understand WHY George goes to such lengths to protect Niles.

Amy / Louise

While the male protagonist is somewhat similar between the two, the female protagonist is completely different. There are some strands that are common between the two, but more than that, it’s really hard to puzzle out what else is common.

Amy, played by Kate Bosworth, was the high school sweetheart. She always wanted to get out of Blackwater. She finally did and went into the world and did an NBC show that everyone in Blackwater watched. She’s larger than life for all of them, she’s the girl who made it. Everyone in Blackwater respects her. But I think that’s not the primary emotion they feel when they see her, the primary emotion is resent. Everyone resents her for having left her hometown. They resent her for coming back to town with an outsider and behave like she’s one of them. Go to the baseball game, go to the church, with an outsider. No one takes well to that. Least of all, her high school boyfriend Charlie.

Amy herself is miffed at the want of passion in her husband. She dresses ridiculously for her early morning runs, she wants her husband to take charge and ask people why they were staring at her. I would actually go as far as thinking that the fact that attracts her to Charlie more than anything is that if she were married to Charlie then NO ONE would have the guts to say or watch her like they do. It’s only because she’s married to this Harvard Academic, who’s never gone hunting, that she’s being stared at and she has lewd comments passed on her.

Her outrageous stunt in the movie where she flashes to Cawsey, Scutt, Bic and Charlie is one of the least subtle ways to strike a match. While Charlie’s been getting that vibe, he’s not yet decided anything but after this scene, where he is clearly surprised and shocked, he realises that she wants her (and in that moment, she probably did want him) and decides to take things into his own hands. The rape follows. Surprisingly, Norman is in on this also and I don’t really know why Charlie lets him, but I guess it’s some sort of punishment for Amy going out of Blackwater and marrying and coming back.

Louise, in the books, is a different character. A caring mother, all she is miffed with is the lack of passion. The line in the David section, where she wants him to show some passion, it’s clear that she thinks all of George is an ACT, an act he puts on for the world, an act that he himself hides his true self behind. She wants to get at him, she wants to provoke him into showing some color.

The first time George can bring himself to slap her, during the Siege when she’s acting insensibly and not doing what he tells her to do, she uses the word relieved right after being hit. That’s what she wanted him to do. She wanted him to TAKE CONTROL, so that she could stop feeling like the one who was making all the decisions. This is some kind of post-moving-to-England trauma stuff where the move was so stressful that all she wants to do now is lie around and not do anything, while her husband takes care of the house.

Louise is most certainly not as simple as that and I have oversimplified the matter. But, that’s what I take away from her character. I think every reader would have their own take-aways for a character like her!

Minor differences

It’s boiling water in the book and boiling oil in the movie
When George first hurts Cawsey and ties him to the window, he does it with a lot of hesitation because he is conscious of him stepping over the line from anger to violence. He says he despises it but realises the need for it and does it anyway. In the movie though, David uses a nail gun and nails Cawsey’s hands to the wall almost without thinking too much about it. And finally, he snarls at him, “I hope you slit your fucking throat.”
In the movie, Amy shoots Norman. It’s revenge for what he and Charlie did to her while they trick David into going hunting.
Charlie has a gruesome death in the movie. That’s just a titbit of information.

Switching to SuperTab; A new Crichton thriller, WHAT?!

2017-07-02T00:00:00+00:00

Ever since I started using Vim as my primary browser, I have been using YouCompleteMe as the completion plugin. It works fine, it is annoying because it tries to autocomplete everything and sometimes that gets in the way of navigation using the arrow keys. In any case, I was fairly happy with the plugin’s performance because it never really slowed down my editing experience. Until a couple of weeks ago.

That was when I saw that the editor just hung for some amount of time, and it was really trying to find suggestions for the word I was typing. This was really hampering the time it took for me to actually write code. And thus, I set out on my search for a better plugin.

One thing about YCM that is slightly irritating is that it has a compiled component. This compiled component doesn’t build on computers that have less than 1 GB of RAM (Looking at you, $5 a month Digital Ocean droplets.)

I tested neocomplete.vim. It doesn’t have a compiled component and has some really extensive configuration options. That include using <CR> as the key to confirm suggestions. That was rather pleasant, perhaps when I have a computer with more RAM and a better processor, neocomplete would be the option. (It is most certainly better than YCM, simply the lack of compiled component ensures that!)

Coming back to my own quandary, I found this new set of completion plugins that show you suggestions but work on pressing the TAB button. It’s pretty interesting, and I started using SuperTab. I am still reserved about how useful it is, perhaps I will report again after a few weeks of use.

I have also just started reading Dragon Teeth - Michael Crichton. It’s really surprising that a new Crichton book was released less than a month ago, seeing as Crichton himself died in 2008. I haven’t read up on the history but I guess this manuscript was discovered. I am about 50% through the book (one 5 hour train journey ensured that!) and it’s been riveting till now. The strange antiquated habit of journaling really takes on a life of it’s own throughout the book (as most of the narration is either the experience itself or what one of the characters recorded in their journal).

Generating Git Punchcard plot from the terminal

2017-06-22T00:00:00+00:00

Around 6 pm today, I found guanqun/git-punchcard-plot.

At 10:45 pm, approximately 4 hours later (of which I spent rather less time writing stuff, I opened PR #7.

Once I cloned the repository, I realised that you need to put the git-punchcard python script in your PATH for it to start working as git punchcard. I didn’t need any of that, I did really want to see a punchcard though. So, I put it there, (Note 1: Take path from the CLI if provided!)

When I generated the punchcard for a repository, it looked like this:

It looked okay. The opacity was a problem. I might have made fewer commits during those times (this is the punchcard of my blog repository generated using this utility). So, I also wanted to fix that so that the opacity of all the circles will be 1 and the circles will all be black.

Implementing these two was rather easy. (I delved into one Cairo function and searched and found os.getcwd() and os.chdir()) Now the punchcard looked more like the one on GitHub, and pretty good:

I was about to open a PR to the repo, when I came across Issue #5: Timezones. It put forth a rather interesting problem. You have collaborators working from different time zones and you would like to figure out when they work, so you can plan upgrades and also schedule meetings without asking them when they work. (decidedly more cumbersome, but why not?)

This was the part that ate up the most of my time. Understanding the internals of the myriad utilities that Python has to work with Timestamps. There isn’t just one module does everything. Oh no, that would be too easy. Instead, there are atleast 4 modules: datetime, time, email.utils, calendar. Each of them has a few functions and they all seem to seemlessly interop with each other. It’s rather confusing and I was finally able to understand what functions to use after considerable amount of moving around the documentation pages of all these functions and an open Python shell. The code I wanted was:

formatdate(mktime_tz(parsedate_tz(RFC_TIME_STR)) + FINAL_OFFSET)
# -> This would give me a RFC 2882 compatible string

One final thing I added in because almost as soon as I had done all this, I realised that the punchcard was for the whole project lifetime which could be years in a lot of cases. I needed some way to limit the period from which the punchcard was being built. One look at git-log’s commit limiting documentation made it obvious to me that tunnelling options to git would be the most flexible option to go forward.

The list of options that git-log supports is rather extensive!.

That was the path I took to get the following punchcard of my 100 Days of Writing project on this blog:

Anyway, the open PR is now on Github at #7. If you have the time, download the script on that branch and make some punchcards for repositories that you work on or others work on. Maybe you will discover something interesting about your colleagues.

Setting milestones to understand TLS

2017-06-13T00:00:00+00:00

I downloaded the TLS IETF draft. It’s 104 pages long, That’s as big as Rita Hayworth and Shawshank Redemption.

This is perhaps my 100th attempt at reallt understanding the internals of how TLS works. My end goal is simple here: I want to know how a secure channel is established between a server and a client. I don’t want an apples-and-oranges-are-fruits level understanding of it, I have that. I know all the digital signature and public key cryptography analogies that are out there. I want to be able to form a general idea of HOW each step works, why it is required, and how that particular step fits in the larger scheme of things.

I started with Wireshark and captured a default HTTPS handshake. I was able to do this rather easily and create a pcap file. Now I atleast have that, which is the ACTUAL packets going through the network.

The order of messages is clear to me but I have decided to take it slow and understand each message thoroughly before trying to put the pieces together.

If I was to give an analogy for this, like those that people always resort to while explaining crypto to others, it would go something like this: I want to understand what is there on each piece of the puzzle before trying to put it together.

First milestone: Client Hello and Server Hello. After this comes the more intense cryptographic part, but these two are important for the decision of which cipher suite to use. (my computer’s least favorite cipher suite uses 3DES, remember that?)

More on this series, hopefully. I am going to print some pages of the spec out because that is probably the only way to keep my sanity and still learn something useful.

Some random thoughts:

In the Hello part of the exchange, both parties agree on four things: the key exchange + signing algorithm, the bulk encryption algorithm, MAC function and a pseudo-random function (PRF). Why are these used? Under my current model for learning about TLS, at this stage, I am going to have to take for granted that these four things are needed somewhere down the line.

Learning React -> Understanding why Redux -> More time spent!

2017-06-12T00:00:00+00:00

So, I had this idea for an app. It’s a pretty basic app and it takes the current location and prints some distances. It does all of this stuff without an active internet connection because GPS DOES NOT NEED INTERNET. It drives me crazy that Google Maps won’t download the area that I live in (which they might know because I occasionally have to turn on Location Services on my phone) or any location that I tell it to.

From their POV, it’s probably not the best because most of the traffic related data on Google Maps is crowd sourced and you ain’t getting no data from a phone that is offline.

So, I decided to leap frog that and build a simple app that would show you your current LAT-LONG, store a list of some LAT-LONGs and then show you distances to whichever places among those that you ask for. I decided to build this in react because getting the current location in react is simple. REALLY SIMPLE: navigator.geolocation.getCurrentPosition((position) => { ... }, (errCB). I am sure I would have lost about 3 hours downloading and getting gradle to build a hello world on my Linux machine! THAT friction right at the beginning of the road is really irritating.

But choosing React was not the ideal thing to do. I had heard enough about Redux to know that it was a good thing that came in handy almost always. A global state, you can add actions, your reducer will take care of it, etc. IT was all very hi-fi and I believed (rather stupidly, I admit) that I won’t need any of that and that I could get away with it this time by putting everything in a single component.

Moral: I finally ended up having to send data / emit events in one component and have the other component subscribe to them. I couldn’t find any reliable event emitter for React Native, which is rather strange but understandable. Consider too that I spent about 2 hours finding an appropriate component to select multiple items in a list. I was unable to and had to settle for a search and add one-by-one approach. Atleast, I was able to find the search and filter plugin easily.

So, the app is in a limbo state where it doesn’t really work and won’t until I either fix this issue or rebuild the whole thing in Redux. I am sure that rethinking the whole idea of the app and it’s component structure in Redux will take up more time than actually writing it down because very much like MVC, Redux ends up structuring the app into independent self-sufficient components that have no clue about what else is going on in the application.

One good thing that did come out of this excercise was my understanding of why Redux is important and how it makes making even the simple apps simpler and easier to design.

Note: I built the app in React Native actually, so all of the references to React above are simply the “idea” of state and building the pure function render() that works against state. The code for the app isn’t open source yet, I intend to make it available once it has reached a more complete state.

Old Habits Die Hard, Criminal show - GET IT

2017-06-01T00:00:00+00:00

I find myself thinking about what to write every day. This has been happening everyday since the end of the 100 days.

A general update, is what this post is.

I am reading Apt Pupil. The book has gone literally nowhere since I started it. Since this is a book of four stories, I don’t even know how much of it I have read, it must be 20-30 pages and the book is slowly progressing somewhere. This initial period of intrigue that exists right before the explosive event and the chase that happens in the end + climax of the book seems to be a common theme in Stephen King books.

I have also been raving about the Criminal show: a podcast which I found on Radio Public. The show is incredible! I have always liked short audio stuff like this, because the recording is done in a studio and once the sound is mixed, the output is incredibly personal. It feels like the person is talking to you through the phone, on a really good connection. Episodes are also 20 minutes which is my daily commute time! There is also something deeply interesting and intriguing about the stories that they pick.

I am also discovering new content on Radio Public, there seem to be several good shows. (99% Invisible is another good show. I am listening to episode 229: The Trend Forecast right now!)

I must admit that I am relieved about not having to write every day!

Day 100 - Shawshank Redemption + The Last Post in this series

2017-05-28T00:00:00+00:00

HELL YEAH, WE ARE FINALLY HERE.

(Prologue: This is going to be a normal Ep 100 post. My thoughts about this series, the things I wrote about (and didn’t), the days I really did write well etc will all be covered in a different post.

That is Rita Hayworth. She was the one that Andy Dufresne began with. For 27 years he toiled, and made it out. It’s an amazing novella! I kind-of knew the ending but was blocking it off so I could still enjoy the book. The revelation is not the climax and I think that’s where the story really excels. In telling the story of two men, vastly different from each other, but stuck in an unfavourable situation and still trying to make their way through life despite it.

It also covers all the problems with the prison system, how it is designed and what happens to incarcerated men after they are left outside, in the world. This is a recurring theme that I have read about in a few books now. I remember George Orwell’s short story about a travelling nomad living and sleeping in different beds each week. I remember Down and Out in Paris and London, again by Orwell, which talks an insane amount about how the poorest people lived in the 20th century, a bare 70-80 years ago. This is even slightly touched on in Malcolm X’s autobiography.

Here’s my goodreads review of the book. It’s rather short at about 140-150 pages and I was able to finish it off during a hectic week by just reading everything on my phone and not waiting to settle down and read. I think that’s the only way I can keep up with reading and also manage work and life despite it. I absolutely must use every single moment of my waking time. It’s cumbersome to do that, but Onward and Upward, that’s the only way I can go now.

I now have two completely new albums that I have discovered (led to, rather) and like. The first one is Silverstein’s This is How the Wind Shifts.

The second one is the album containing my recent favourite. Incubus’s If Not Now, When?

I read Incubus’ wikipedia page, and I was really surprised to find out that the band was formed in 1991 when they were in high school! The 90s were indeed a different place. I wonder how the high school bands of today are doing.

I am back to listening to Humbug now, in fact Cornerstone is playing the background as I write down the last few words of this 100 day post series. It’s a bitter sweet moment. I am relieved to have finished off a challenge that I took on on a whim and didn’t give much consideration to. I slightly panicked when I realised what I had committed to and wanted to get out. I settled down after about 3 weeks and wrote about whatever I could find. I struggled for content and had to make do with mediocre posts. And through all that, I still wrote a 100 blog posts in 100 days. This is BIG for me!

POST #100 is OVER

Day 99 - Mongoose + async act up to create cocktail

2017-05-27T00:00:00+00:00

I found another really strange thing that happens with Mongoose when used inside Async.

let mongoose = require('mongoose');
mongoose.connect(db_conn_string);

let util = require('util');
let async = require('async');
let Schema = mongoose.Schema;
let ObjectId = Schema.ObjectId;

var PersonSchema = new Schema({
  name: {
    type: String
  },
  username: {
    type: String,
    required: true,
  }
});

var Person = mongoose.model('Person', PersonSchema);

var p1 = new Person({ username: "u" + Math.random(), name: "User 8" });
var p2 = new Person({ username: "u4" + Math.random(), name: "User 10" });

async.auto({
  other: (callback) => p1.save(callback),
  normal: (callback) => p2.save(function (error, created) {
    if (error) {
      callback(error);
    } else {
      callback(null, created);
    }
  })
}, function (error, results) {
  console.log(require('util').inspect(results, { depth: null }));
  process.exit(1);
});

This code gives the following output:

{ other: 
   [ { __v: 0,
       username: 'u0.16941186863323354',
       name: 'User 8',
       _id: 5929cc51d66b3e0a4c4796f0 },
     1 ],
  normal: 
   { __v: 0,
     username: 'u40.00008901152893536768',
     name: 'User 10',
     _id: 5929cc51d66b3e0a4c4796f1 } }

If you have not noticed the inconsistency yet, see the results again. Two apparently same blocks of code seem to be doing something completely different. One returns an object while the other returns an array containing an object and an integer. (On further investigation, I found out that the number is the number of people that were created. If you do p1.save(...) and then again p1.save(...), the count turns out to be 0)

Now, other is the new function notation that has been activated in ES 2015. I have read the syntax documentation for this and nothing seems amiss here.

Trying this out without async, using code that looks like this:

// Mongoose connection, Person schema, Person model declaration etc

function genPerson({ username, name }, done) {
  let p1 = new Person({ username, name });
  p1.save(done);
}

genPerson({ username: "u" + Math.random(), name: "User" }, (error, created) => {
  if (error) {
    console.error(error);
    process.exit(1);
  } else {
    console.log(created);
    process.exit(0);
  }
});

Inside of this, created is the object that was created an NOT the array that is returned in the previous code’s other case.

A very intriguing issue that has now been put out there in the public for people to find and explain. This undoubtedly restricts the usage of the arrow syntax with async unless you would like to have random [0]s everywhere in your results.objName[0] references.

POST #99 is OVER

P.S. 99 posts are OVER! YAY YAY YAY! One more post to go tomorrow and I would have finally completed 100 posts in 100 days. More or less writing daily.

Day 98 - Rains a bit, Floods a lot in Koramangala

2017-05-26T00:00:00+00:00

08:00 pm: Walk into a restaurant for dinner with a few friends and seniors from IIT KGP.

11:30 pm: Decide to step outside and be faced with one feet of water in every direction the eye can see.

Reaction: WHAT THE FUCK HAPPENED HERE?

The best reference to come up with here is: July 26, 2005, Bombay

I was there, 9 years old at the time. Hardly understanding what the fuss was about, schools weren’t there for a few days I was definitely real happy about THAT. There was a lot of stuff going on at that time in the city, I got to know of most of them after the fact. As stories that were recounted several times whenever the topic of flooding came back.

As I am reading the wikipedia page, I am extremely surprised to see that there were PAPERS written on the topic!

Andharia (2006) contrasts the “widespread acts of generosity and altruism” in Mumbai with the general social disorder that was seen in the aftermath of Hurricane Katrina in New Orleans.

Well well well. I don’t want to read that paper because it is probably blabbering on and on about Indian culture and tradition and the helping tendency of Mumbaikars etc. There’s nothing worth being in a paper there. And especially, comparing to something like a Hurricane in the US which was properly managed is unfair.

12 years on, I waded through knee deep water for about 700 m before we were on High ground again and able to reach home safely.

POST #98 is OVER

Day 97 - The Circle - complete review

2017-05-25T00:00:00+00:00

I watched The Circle. It’s a movie everyone needs to watch

The movie touches on some of the core concepts of privacy and does a really really good job of putting a clear LINE where transparency begins and privacy ends. They do it in an obscure fashion with no direct references, main characters who are incredibly skewed in their personalities, visions and with motives unknown to the audience. Money might be a driving factor, but Tom Hanks’ character and the guy Tom Stenton, they have LOTS of data, I wonder if money is their only over reaching goal.

Anyway, putting all of that aside, it’s a great cautionary tale of what happens when people start believing that what’s not on the internet and events whose memories are only held in a person or a few people’s memories might as well have never happened because there is no record of them!

SATIRE! I want to call this movie a satire, but the typical absurdity of satire and the explicit references to things that are outlandish and obviously impossible is lacking. The ending is very dystopian and 1984. (I wonder if the writer / director were influenced by that book. The movie ends with people not having telescreens forced on them, but voluntarily putting cameras in front of them for the whole day. They are allotted time for bathroom breaks of 3 minutes. “Bathroom lighting is the best lighting a selfie can ever have” - A wise woman once discovered.)

After 60 minutes of the movie, the tide was slowly turning and all the ominous events of the second half of the movie were on the horizon. As the movie stopped for the interval, and the lights came on, one man summed up this movie in a couple words:

Scary, huh?

The first half of Mae Holland is where most people are at right now. Not too famous, overwhelmed at how many people are interested in their daily activities which to them have been routine and mundane forever, not too worried about who is watching what, not realising that they have lost privacy simply because they have location turned on on their phone, or post geotagged photos to facebook or twitter unknowingly. (Twitter used to strip away all location information from photographs, I saw a TED Talk about this once. I don’t know if their policies have changed now though, I hope they still strip information by default and make sharing location a little bit harder so people explicitly have to do it)

All of this said, the ending is a bit confused. She says “Secrets are lies” in the beginning of the movie, she releases all of her bosses’ emails, everything is out there in the open. But the ending shows her waking up, saying good morning to the camera and then zooms out to show a collage of millions of other camera frames that look very similar. They outed the bosses, but they still couldn’t stop the loss of privacy? I don’t get it. I will definitely watch the stream the movie and watch it again, maybe things will be clearer then.

A special mention for Karen Gillan’s performance of Annie’s character! The make-up, her disheveled hair during the middle part of the movie, the wrinkles under her eyes and her final “going-back-to-normal” once she goes back home is a treat to watch on screen!

POST #97 is OVER

Day 96 - Shawshank Redemption - getting to the end

2017-05-24T00:00:00+00:00

The second type of guy knows that there is no harm in hoping for the best as long as you are prepared for the worst

(p. 88, Rita Hayworth and Shawshank Redemption – Stephen King)

I have known this particular phrase / quote for a long time now. I finally found the source (or it could be older than even this book!) In any case, it is a GREAT quote to live by.

(27th May, 2017) I have now read about 100-110 pages of the book. Dufresne has made the escape, Red is merely tying up all the knots. I have read the whole book (almost) on my phone, and I didn’t check where the book ends. So I am going by the rough metric that this book is about 180 pages long, I could be horribly wrong. I would prefer for the book to be shorter. Not because the book is bad (IT’S NOT!) but because a book ending before you expected it to is rather convenient! (considering that I am two books behind schedule, I would like to keep it that way atleast and not fall back even further!)

The book has been surprisingly graphic and I am glad I didn’t watch the movie. I will probably not watch the movie even after because I know the story and only the gory details of him coming into the prison, the sisters etc will be new to me.

(24th May, 2017) I am booked for a showing of The Circle tomorrow night. It has been rather shady movie till now. Except for the fact that Emma Watson joins a company led by Tom Hanks and that the company is doing something with incredibly small cameras and a speech that ends with

Knowing is good, but knowing everything is better

I know nothing about the premise. Alien: Covenant was the exact same thing to me, I hadn’t even read the plot synopsis anywhere before going to the movie because I thought it would give away too much. I have watched The Circle’s trailer and it’s suspiciously less tech and more gimicks and punch lines. Companies that do that are generally upto no good. Detailed review of The Circle coming up tomorrow.

POST #96 is OVER

Day 95 - BEGIN Shawshank Redemption, another query population hurdle

2017-05-23T00:00:00+00:00

UNFULFILLED POTENTIAL

If you have seen any trailer of The Circle (2017), you will know that this is the thing that Emma Watson’s character is most scared of. I just finished watching Good Will (Goodwill?) Hunting. That movie is the portrayal of unfulfilled potential. The ending is good, in my opinion though, the movie is over rated generally.

I have gone through 2 pages of Rita Hayworth and Shawshank Redemption. It’s pretty easy reading (after that nightmarish historical mystery book, I bet anything with actual people walking around the place would be easy reading) and I know who Andy Dufresne is. Fair enough, more updates soon.

I went back and had a look at the “Should we send a notification?” flow chart that was recently tweeted by someone at Slack. It was a quip at how people say that “They could build X on a weekend”, X being anything like Facebook, Twitter, Slack etc. i.e. Deceptively simple products that have had a lot of thought put into their design, but no one acknowledges them. Worst of all, even other engineers don’t.

When Hacker News commenters say "I could build that app in a weekend!" I think of this chart of how Slack decides to send a notification. pic.twitter.com/LopicAyzkL
— Matt Haughey (@mathowie) March 3, 2017

Another query population hurdle that I ran into was the population of an array of Strings, which are a unique field in another collection. Let me explain:

Person: {
  username: {
    type: String,
    unique: true,
    required: true
  }
}

Comment: { ...
  usernames: {
    type: [String]
  }
}

I have read the mongoose query population document several times but this particular case it seems hasn’t been addressed. If it were an array of ObjectIds, of course it would be a simple populate('usernames') addition to the query.

Virtuals was something that I tried to look into but that again serves another purpose altogether. I will keep looking but I think mongoose doesn’t support something like this right now.

POST #95 is OVER

Day 94 - Starting Shawshank Redemption + Good Will Hunting

2017-05-22T00:00:00+00:00

Next up: Rita Hayworth and Shawshank Redemption by Stephen King. I read my Stephen King book a few months back, and I promised to read more Stephen King this year. Different Seasons (the set of 4 novellas) is the perfect way to get started. All the four novellas are small enough for me to be able to be confident enough that I will finish each soon.

(It’s also a convenient way to cheat through the Goodreads reading challenge and mark 1 HUGE book as four books. I don’t know of any other Stephen King book that would afford that particular oppurtunity.

Good Will Hunting is the next movie I would like to tackle, I didn’t finish watching it just yet. It has been a great movie till now, especially Skylar’s character and the actress playing her. With her British accent, and her studying to be a doctor, Matt Damon is simply chicken-ing out of the relationship that could probably mean a lot more, if he would just try. I hope they get together by the end of the movie.

I also see that Mike Ross’ character in Suits is heavily inspired from Will Hunting. Both of them had a rocky childhood, can immediately assimilate a lot of pages of material and understand it. One line that Mike Ross had in Suits rather early in the season when everyone was so awed with his ability to remember everything that he saw once: “Once I see something, I understand it. Once I understand it, I never forget it.” It is a shame that show went to the dumps just 6 seasons later. Come to think of it, after the initial awestrucked-ness of everyone with Mike’s abilities and Harvey’s swag (for lack of a better word?), there was nothing propelling the story forward except for some petty (in retrospect) office politics.

After Mike’s secret is revealed to the world, there is absolutely nothing to go forward with. They should have ended the series with him going to jail. A lot of other people agree with my assessment of this series. Have a look at the viewership graphs on Wikipedia for Suits through the seasons. The drop from season 1 and even season 2 and 3 is noticeable.

POST #94 is OVER

Day 93 - Watching Alien Covenant + Strolling on M.G Road

2017-05-21T00:00:00+00:00

I finished reading Daughter of Time today! -> Goodreads review

Today will be one of the few days that I have been able to post well before midnight AND on the same day as the post is due to be out. I think as this endeavour comes to an end, I would like to be able to make the last 7 posts on time. Ending on a high, is what it is called, I think.

I watched Alien: Covenant today. It was so totally on a whim. It’s a GREAT movie though. Saying anything more about it will almost certainly spoil the plot for the reading-between-the-lines sort of readers of this blog.

Michael Fassbender does an amazing job playing a robot (“synthetic human being”). One question that comes to mind is whether an actor who can play a synthetic with apparently no emotions is a GREAT actor or a really bad one.

One line in the movie that can be divulged without spoiling the movie, as long as no context is provided is:

serve in heaven or reign in hell?

As is perhaps obvious from my writing that line in a heading block, I absolutely loved this line. I am sure this wasn’t the first time that this line has come up. But it really struck me when it was delivered. There are a lot of archaic references, both to literature and music, in this movie. There is something about robots and archaic references, I have seen them often now.

Shutter Island remains unwatched. Surprisingly, I fell asleep EXACTLY at the same point I fell asleep when I was first watching the movie. I distinctly remember the scene as being the one in which the two detectives were orderly’s clothes and walk into Ward C and there’s this water dripping or something as they walk through a hallway.

That scene is so unsufferably devoid of mystery and despite that, it’s a SLOW SLOW scene where things don’t come to a crescendo soon enough. I wonder what people who love the movie think about the scene. The beginning of the movie, till that scene, has been ominous and something seems off. (Especially about the orderly who puts Leo to bed after he has the weird so-called “migraine”)

Now, I intend to find a new book to read.

POST #93 is OVER

Day 92 - Shutter Island Rewatch + Departures - Silverstein

2017-05-20T00:00:00+00:00

Shutter Island vs. Inception. I immediately picked Shutter Island, ironically, I know Inception’s story and even the little subtleties of the screenplay, storytelling, dialogues, the flights, the characters, the levels etc etc way better than Shutter Island which I watched only once. I decided to fix that and watch the whole movie again.

The Circle is out. I wanted to read that book before the movie came out and then I could go to the movies and watch the movie and decide what I liked better: A few hundred pages or Emma Watson and Tom Hanks. I decided to do that one year ago with Great Expectations (Dickens) and Fitoor. I managed to finish the book and watch the movie in one of the last days that it was in the theatres in Calcutta. This year, it was not to be. One of the Gillian Flynn novels is coming out as a TV series soon, THAT should be interesting.

I have found one more song that I can obsessively listen to. Departures (acoustic) by Silverstein. I have nothing against the rock version of this song, which has a little bit of growling and lots of electric guitar. It’s just that I like the acoustic version better because of how it manages to convey the mood of the songwriter despite being so obviously deprived of instruments.

I am going to go back to watching the movie and then reading Daughter of Time (maybe) and hopefully, finishing it this weekend.

Also, this is today’s post, being posted today. I am finally back on track, with 7 more posts to go. The time from 18th February to now has gone by rather fast, it’s been 3 months and 2 days since I started this challenge!!

POST #92 is OVER

Day 91 - Daughter of Time disoriented me, so a Glossary

2017-05-19T00:00:00+00:00

The Daughter of Time is a slight disorienting book. It is about a historical murder (or not?) and dates back four centuries. The names are all extremely confusing (of the characters in the book that the main character is reading). For now, Alan, the Matron, the two nurses, Marta, Brent and Laura seem to be the only ones in the main story that matter. There’s also a Sergeant, and a Super who never enters the picture (Thankfully!)

On the historical side though, things are murky as hell. Richard III is the character of interest, he has an elder brother by ten years Edward. I don’t know what number this Edward was though, IV seems like a good guess. He also had another elder brother called George whose children were not given the throne because someone said that they were illegitimate. Richard had a wife who was one of his neices. A sister to the two litte boys who have apparently been murdered by Richard III.

Some other characters that have made an appearance till now are Cecily Nevill, Richard and Edward’s mom. A Woodvill woman who was Edward’s wife and all of her relatives who are all upstarts.

I also know that the circumstances surrounding Edward’s death and Richard’s actions right after his death were all textbook good younger brother and there was no reason to believe that he had any motive to murder his dead brother’s only two sons just then. They investigation is still on (and will remain so for about 47% of the remaining part of the book).

I wonder if anyone uses Foursquare anymore. Back when I installed it, it was a really good looking app. Now, it’s a normal looking app but it took me to a great Chinese restaurant somewhere in the middle of Koramangala yesterday. Hunan, it was called. Definitely recommended.

(Hunan was also the name of the restaurant that Jerry, George, Elaine, Kramer and the gang go to in Seinfeld in NYC in the 90s!)

POST #91 is OVER

Day 90 - Goodbye Capture Groups, Hello Word Boundaries

2017-05-18T00:00:00+00:00

There was an interesting Regex problem to solve today. Insert commas in a number according to the Indian numbering system. There’s an inherent asymmetry in the Indian system

The number: 1 million

Indian: Rs. 10,00,000 American: Rs. 1,000,000

The first approach that I took was to use capture groups. Capture the last three digits, put a comma in front of them, then the next two and so on. This would have worked if all the numbers that this replacement needed to happen on were of the same number of digits. That kind of thing never happens in real life.

Scouring the internet, I eventually landed on the Regex used to insert commas according to the American system. The regex looked like this:

str.replace(/\B(?=(\d{3})+(?!\d))/g, ",")

I know a fair amount of Regex from having used sed inside and outside vim regularly to replace text. This regex had a lot of things that I didn’t recognize. \B, ?=, ?! and also some parentheses which weren’t working as group capturers and a comma appearing in the middle of two characters. I had never seen that before.

Then, I found this website which has incredibly detailed articles about all the operators that Regex supports and the things that can be accomplished with it.

Armed with the knowledge of what “Not a word boundary \B” and Lookahead assertion using ?= means, I delved into finding a Regex that would solve the problem.

The solution turned out to be a two step:

str.replace(/\B(?=\d{3}$)/g, ",")
   .replace(/\B(?=(\d{2})+(?!\d),)/g, ",")

The first one finds the first NOT a word boundary which is succeeded by 3 digits and the end of the string

The second regex looks for NOt a word boundaries that have exactly two digits after them and stop matching when a comma is encountered.

All of this magic (not really.) is possible because Lookahead assertions don’t consume characters. Consuming characters basically means that the regex pointer moves forward from one character to the other after looking at it. Lookahead assertions keep the pointer at the character being considered for a match and simply lookahead and assert or not assert the stuff they have been asked to.

I would love to have a look at how this is implemented in sed or even C++ for Node, but I am unreasonably sure I wouldn’t understand the implementation. This “looking at the implementation and not understanding it” will become a post inspiration in the future.

P.S. All the code above is JavaScript, duh.

POST #90 is OVER

Day 89 - 4 chapters into Daughter of Time. Where is this book going?

2017-05-17T00:00:00+00:00

I have read four chapters of Daughter of Time now. It is going good till now, I have absolutely no clue about the history or England. A cursory reading of the Wikipedia pages of House York and Richard III himself, took care of that (partially). The story itself is going to probably revolve around Richard III, his brother Edward, the two nephews, Tyrell and two of his hench men.

The way everyone has a new story or a new view point to add to the story of Richard III is totally in line with how History is as one chooses to see it. As a friend of mine put it one fine evening, over dinner, “All objectivity is lost when you are reading history. It is not just facts, dates, causes and effects anymore. It’s much more than that.”

Why everyone keeps coming to visit Alan Grant is not broached in the book at all. This is the fifth book in the series, and I feel like there are some presumptions that were made about the reader’s knowledge of Alan’s temperament. How he was good with faces, or only people interested him and not generalisations. The author doesn’t spend too much time on these points and simply brushes past them.

One final note: The endgame of this book isn’t clear to me. It’s a mystery, one that (probably?) can’t be solved in the old fashioned sense of the word. There can be theories and facts that refute those theories, or there can be theories that don’t have any facts refuting them, but none proving that that was indeed what happened. So, where is this book going? That’s an exciting situation to be in. To not know where a book’s plot is headed. A refreshing change!

The Digital Ocean script is almost done now, it can create a snapshot. Deleting the older snapshot though just became a bit more trickier. As it happens, the time taken to create a snapshot is ~20-25 minutes. I never measured it, but it takes a significant amount of time. So, to delete the older snapshot after the newer snapshot has been succesfully created, we would need another cron job, running at a higher frequency that polls for multiple automatically created droplets and deletes the older one if found.

All of this so that billing can be kept at a minimum. I am writing the scripts in Ruby, the second script should be easy enough. I have decided on a pattern for naming the automatically created droplets, so as to not accidentally delete intentionally stored droplets at any point in time.

POST #89 is OVER

Day 88 - Query population with refPath works. What went wrong?

2017-05-16T00:00:00+00:00

I have been delving into Mongoose and Lodash rather heavily in the past few days. A strange thing that I found with Mongoose Query Population was that the dynamic referencing of the model to populate from, to be done using refPath in the Schema, didn’t work for multiple documents. (or so I thought! Read on.)

The same thing worked for a single document. I decided to conduct an isolated test to check if this was a problem, or if we were simply using it wrong. The test worked well, it isn’t a problem with mongoose. (I am still no closer to figuring out what the anomaly in the original use-case was)

This was the javascript for the example:

let db_conn_string = "mongodb://<host>:<port>/<db>";
let options = { user: "<username>", pass: "<password" };

let mongoose = require('mongoose');
mongoose.connect(db_conn_string, options);

console.log("Connected to mongoose");

let util = require('util');
let async = require('async');
let Schema = mongoose.Schema;
let ObjectId = Schema.ObjectId;

var PersonSchema = new Schema({
  name: {
    type: String
  },
  username: {
    type: String,
    required: true,
    unique: true
  }
});

var Person = mongoose.model('Person', PersonSchema);

var p1 = new Person({ ... });
var p2 = new Person({ ... });
var p3 = new Person({ ... });

var people = [p1, p2, p3];

p1.save((err) => { ... });
p1.save((err) => { ... });
p1.save((err) => { ... });

var ChatMessageSchema = new Schema({
  message: {
    type: String
  },
  author: {
    type: ObjectId,
    ref: 'Person'
  }
});

var ChatMessage = mongoose.model('ChatMessage', ChatMessageSchema);

var c1 = new ChatMessage({ ... });
var c2 = new ChatMessage({ ... });
var c3 = new ChatMessage({ ... });
var c4 = new ChatMessage({ ... });

var ReportAbuseSchema = new Schema({
  message: {
    type: String
  },
  resource: {
    type: ObjectId,
    refPath: 'entity'
  },
  entity: {
    type: String // Can be either Person or ChatMessage
  }
});

var ReportAbuse = mongoose.model('ReportAbuse', ReportAbuseSchema);

var r1 = new ReportAbuse({ 
                            message: "report a chat", 
                            resource: "5798...", // This is a comment ID
                            entity: "ChatMessage" 
                        });

var r2 = new ReportAbuse({ 
                            message: "report a person", 
                            resource: "5905...", // This is a PERSON ID
                            entity: "Person" 
                        });

records = [c1, c2, c3, c4, r1, r2];

let iterator = function (record, callback) {
  record.save(function (error) {
    if (error) {
      console.error(error.stack);
      process.exit();
    } else {
      console.log("Saved record!");
      callback();
    }
  });
};

async.each(records, iterator, function (error) {
  if (error) {
    console.log(error.stack);
  } else {
    console.log("SUCCESS!");
  }
});

I ran the script once. Opened the DB and copied over the user IDs to put them in the comment and reportAbuse record creations.

The above code didn’t work as is. There was an extremely weird $author_1 dup error. Now that I look at the code again, I wonder if it is because there are a few parallel save operations happening on the Database. I can’t be sure just yet, more updates to follow in the next few days. (I think)

I ran it just by creating one comment, one person and 2 report abuses. And then querying all the report abuses and asking mongoose to populate resource. It worked as expected.

let query = ReportAbuse.find().populate("resource").lean().exec(...)

POST #88 is OVER

Day 87 - Metakgp Wiki goes down and comes back up; Script to snapshot each week

2017-05-15T00:00:00+00:00

Next up: Daughter of Time by Josephine Tey. I was going to read Dennis Lehane’s Mystic River (because Donna Tartt said that that was the book that pushed her to write Secret History), but it was about 350 or 450 pages or something, and I wanted to stick to smaller books which I can be reasonably sure I would complete than involve myself in a huge book which I would be rather skeptical about completing.

The setting for Daughter of Time is peculiar, apparently, it’s an analysis / investigation of a painting and a murder by an investigator on his hospital bed for 2 days (or more? The whole book, basically). I am a little nervous about whether I would like this book or not, the lethargy that is conveyed in the plot synopsis is part of the reason why. I am taking the plunge, nonetheless. (That book also featured in a major daily’s Top 100 Crime Novels list)

Metakgp’s Wiki went down some time around 2017-05-15 0930 +5.5. It was down for about 40 hours before being bought back up with a recovery read-only server and appropriate changes to the DNS records.

It was because of a kernel panic, it seems for now. We are tracking the incident on a issue at Github: metakgp/metakgp #17. One thing I was surprised with was the speed with which the update in the DNS records started showing up on my browser. (It was immediate) I wonder if that was because Cloudfare was doing some kind of magic with DNS propagation or because the browser got the correct DNS instead of the cached ones because of the earlier 500 responses from the cached one.

This also led to metakgp/metakgp #18: Snapshot the wiki droplet every week. The Digital Ocean API is rather powerful, and I have heard a lot about how people use it to add and remove droplets from their fleet depending on the load to their application. This particular functionality should be possible with a POST request and a DELETE requese. I am guessing that from the REST API assurance that DO seems to imply, I think it should be easy enough to write the script.

I have started writing it in Ruby and some part of it is done, I should be able to finish it off and deploy it this week. I have found that small tasks like these have an exponentially decreasing graph where the y axis is Enthusiasm to finish the task and the x axis is Time after ideation. Right now, the the x-axis is at 14 hours and my enthusiasm is above the threshold. We will see what becomes of that tomorrow morning.

POST #87 is OVER

Day 86 - Cruising Attitude - A list of the things I learnt about flight atttendants and flying

2017-05-14T00:00:00+00:00

I finished reading Cruising Attitude today! It was a really easy read, with lots of light hearted humor and some really gruesome experiences, both of the author herslef and her friends, who play a major role throughout the book.

There are a lot of things about the flight attendant profession that I learnt from the book, I can’t make an accurate, exhaustive list of them, but I will try to make some list that I can refer back to, later:

They get paid far less than everyone thinks they do
They have to bring their own food from home because the snacks on a flight are not for them and they can eat first class food, ONLY if it is left over and if the pilots don’t want it (some crews might not even offer the pilots food!)
A turn-around is a trip in which you go from one place to another and return on the same day
They try to save money everywhere, including the places they stay at while they are in a city
Seniority decides what trips you get to make, and what places you get to server within the same flight
Flight attendant training is a highly stressful few months in which they teach everyone everything that a flight attendant needs
FAA minimum crew is the minimum number of people that MUST be present on a flight. ( ~ 1 attendant per 50 passengers)
Captains sit on the left side of the plane and they get paid a LOT more than First Officers
The Captain and the first officer are served different food, in the off chance that the food was poisoned. The captain gets to decide what food he wants to give (from the offered ones) to the FO
Seniority is everything in the pilot world also. It decides if pilots have to fly before, on or after holidays, what kind of trips they have to make, what side of the plane they sit in and what food they get to eat
Turbulence is of 4 categories: light, medium, extreme, severe. Turbulence is rarely rated as severe
DO NOT press the call light unless it’s an emergency. Flight attendants appreciate if you flag them down while they pass by you or walk up to the galley to ask for coffee or the like
Each flight attendant is required to walk through the cabin once every 15 minutes.
Food is almost always served from the front of the plane to the back, with some exceptions (which were later phased out? I am hazy here.)
When flight attendants use their passes to travel, they are non-revving (Non-Revenue Travel)
Non-revving attendants are always on standby until the last moment when that it becomes clear that a passenger is not going to be able to make it to the flight anymore
As the plane lowers, air pressure increases and this pushes the ear drum inward. “Chewing gum helps, as does swallowing or yawning during descent”. Blocked Ears are a common reason due to which flight attendants are grounded. Continuing to fly with blocked ears, can cause severe damage. #
“Airplanes fishtail during turbulence, making the back of the plane much bumpier than the front. So, one good trick is to book a seat as close to the cockpit as possible.” – p.201
On long flights, it is a good idea to take sleeping aides (like Ambien) AFTER take-off. In the off chance that there’s a mechanical problem, and everyone has to move to another plane, doing that without Ambien is more conveient.

I went through the whole list of my “Notes” section on my Kindle and it was rather small, for a 260 page book. I didn’t highlight a lot in this book, I wonder if that is because of the care-free, amateur writing!

This book provides a rather unique biographical perspective into the lives of flight attendants and I definitely recommend it if you have a few spare hours. Perhaps when you are passing through an airport? The setting would be near perfect.

POST #86 is OVER

Day 85 - Devastating global ransomware attack(??), Node.js event loop

2017-05-13T00:00:00+00:00

A global ransomware attack and the oppurtunity to see it play out in real time? Wow, that’s scary and exciting, no offence to everyone who was affected. It sucks big time, but as someone who is fascinated by how these things start and how they spread, it is particularly of interest to me.

The exploit was a Windows port 445 SMB loophole that the NSA apparently knew about but disclosed to everyone only after their recent leak. Snowden quickly took to twitter to criticise them for reducing the amount of time that everyone had to prepare because they disclosed the exploit’s existence so late.

The way it spread (and the way people were affected) is of more interest to me, because that is what everyone should be on the look out for and not do when the situation arises. Clicking on spam emails and attachments is a security NO-NO that has been re-iterated ever since the existence of Spam emails. As one article pointed out, “It was surprising that NHS computers got affected because NHS employees are instructed not to open spam-looking emails.”

There is some talk about how the worm mainly spread through networks inside businesses where people might have been running outdated software for legacy support reasons.

My internship at Elanic is mainly Node.js backend till now. This talk is a pretty good primer on how Node.js really works. There are some old Ryan Dahl talks about his philosophy when he was building Node.js and why he thought non-blocking asynchronous I/O was the way to go for building web applications as compared to the synchronous PHP way of db.query(...) that was popular in those days. I found his talks to be a bit too dry, and being old, they were also not in HD which made the metrics he was showing rather hard to make sense of.

Nginx apparently runs using an Event loop and uses only a single thread of execution as compared to Apache’s one-thread-per-request model. While that should ensure a significant increase in performance if one switches from Apache to Nginx for a lot of concurrent users on a site, one of the major hurdles is how hard it is to setup Nginx properly compared to the basic configuration of Apache, which is literally just a couple of apt-get install steps.

An nginx reverse proxy to setup Node.js applications seems to be the weapon of choice for people wanting to setup ExpressJS REST APIs, but the scaling part of this setup is something that needs to be managed manually and perhaps providers like Heroku would do a better job at this. At $7 a month compared to Digital Ocean’s slightly cheaper but vastly smaller $5 a month droplet, the difference in costs is perhaps not too decisive at a small scale and would not justify the time that would go into working on scaling servers, compared to developing things.

POST #85 is OVER

Day 84 - Handmaid's Tale Audiobook is here! (+ Claire Danes)

2017-05-12T00:00:00+00:00

I just recieved the 2012 audiobook version of Handmaid’s Tale. AND I LOVE IT.

I have tried one audiobook before: The Hitchhiker’s guide to the Galaxy. I didn’t like the book as much as everyone else told me they did. I don’t know if it was the weird setting, or an immaturity on my part to appreciate fantasy-ish literature. I know there’s a proper word for that, I can’t remember it now though.

I started hearing the audiobook. They have done a good job of telling people exactly how to move the files onto an iPod and I was able to do it in about 10 minutes with an old Windows 7 laptop. (In retrospect, with the Windows SMB ransomware out there yesterday, this might have been a dodgy move. That ransomware was delivered through spam emails though. The post mortem of that attack will be interesting to read)

The first few chapters are very very shady about the plot of the novel, nothing is given away, nothing is explained explicitly. If a reader were to read / listen to the novel without having read the synopsis of the novel, then they would hardly ever be able to guess what the Ceremony entails for Offred, the Commander and his wife.

I remember a lot of the first part of the book, before the revelation about how they got there. In retrospect, a few days after reading the book for the first time, I can actually appreciate how insane it sounds that the Commander and Offred actually left their house and went to a shady night-club-ish place. Things only escalate from there with the relationship between Offred and Nick, in his room above the garage. That plot point sounds like something I imagined. It’s that weird, when you consider the setup of the house, the country and the government after learning about those things.

(I am talking about Handmaid’s Tale being great in the 84th post of this series. DO NOT, for one second, the irony and coincidence of that is lost on me!)

POST #84 is OVER

Day 83 - StartPage search? P.S. I am sticking with Chrome

2017-05-11T00:00:00+00:00

Talking about the tweet that got popular, there was in particular one user, @architex63, who had some radical opinions about another well meaning person who suggested that you might want to use StartPage Search if you want Google’s accuracy minus the Google’s tracking.

@marianstiehler @StartPageSearch @_icyflame @duckduckgo people in the know, know. I know a few of those people. I just try to warn people. can't really say why. do some research. good luck!
— Beeve Stannon (@architex63) May 11, 2017

I tried using classic.startpage.com and while the search results were accurate, the first three results were injected ads! They look exactly like the results and weren’t blocked by ublock. While ads are slightly annoying in general, and extremely annoying when they appear as the first thing that can be clicked in the results of a search, I might put up with them if I could be assured about the security of the search engine itself.

DDG was suggested by Snowden himself, and I tend to trust the guy who risked his life and did incredibly dangerous stuff against arguably the most powerful government just to prove a point to a large amount of people. StartPage search hasn’t had that kind of acceptance just yet.

Chrome seemed to be the common gripe among people, but frankly, I don’t see the angst against that. Brave for Android is a great browser and it’s the best for Android with in built ad blocking and HTTPS everywhere. For linux though, Chrome is the best that’s out there now. I have used Chromium, raved about the low RAM usage of Firefox, and even tried out there ones like Opera and Vivaldi, both of which kept hanging on me and just didn’t work.

I am not going to switch back to anything other than Chrome for a while now, I can’t see what I could switch to. Chromium’s open source version might be an option. For now though, Chrome + uBlock + Ghostery + DuckDuckGo is the best I am going to commit to using daily.

POST #83 is OVER

Day 82 - A tweet that blew up

2017-05-10T00:00:00+00:00

I tweeted something that quickly became my most liked and retweeted tweet.

Surprisingly, I went to the myactivity page because of a button on the DDG search results page that pointed me to a DDG article about privacy measures to take on PCs and Android phones.

As someone quickly pointed out, I didn’t have Google Analytics or other analytics JavaScript running on the client side blocked. I used to use Ghostery for this. They gave a list of the tracking services which are installed on the sites you visit.

An earlier version used to show a list of the trackers and it wouldn’t go away immediately. I don’t think they even had the functionality to block scripts at the very beginning. I didn’t try to remove that list, I admit. I simply uninstalled the extension.

The list of trackers would appear on the bottom right corner, and would be there for the first few seconds of visiting each and every page.

I installed the latest version of Ghostery’s chrome extension and the list has been replaced by a much less visible and less irritating “number” that they show in the bottom right corner of every page after loading the page. Selecting the “Block All” option (which honestly looked like the easiest thing to click on the extension configuration page) blocks all scripts. I can see the scripts complaining in the console.log or Chrome itself notifying the console that scripts were blocked.

There were also some more interesting conversations on the tweet. I have to cover them in the next post. have to xD.

POST #82 is OVER

Day 81 - MISSED; Catching up on writing; Books

2017-05-09T00:00:00+00:00

In perhaps the largest backlog of mine in this series of posts, I am now four posts behind. This post should have been out on 9th May, 2017.

I must run a test about how many times I spoke about this series in a post. I think it must be every other post. I am not even kidding. I am also at a position where I mut write the posts for the next 4 days.

I have moved slightly forward in Heather Poole’s book. Her mother becomes an air hostess and starts working alongside her! That is probably one of the weirder plot points in the book. The back stories and the way her roommates live their lives is rather interesting. I am glad she spent as much time as she did on describing them.

Paula Hawkins released a new book called Into the Water. Megan Miranda is on tour for promoting her new book Perfect Stranger. The female psycho-thriller fiction market seems to be on the up ever since the stellar Gone Girl and the equally good book but slightly less hot adaptation The Girl On the Train.

I am hoping for an imaginative plot with some crazy (and believable) twists.

POST #81 is OVER

Day 80 - Homeland season 1 - episodes 1 to 8 - a COMPLETE RE-WATCH

2017-05-08T00:00:00+00:00

I rewatched the first 8 episodes of Homeland season 1. The character development of Carrie, Saul, Brody and David Estes (whom I totally ignored when I first watched it) is mind blowing.

Specifically, the relationship between Carrie and Brody. After their first meeting, leaving each of them smiling and mystified by the other in the rain, they go for the weekend to her cabin. Here, one of her dialogues makes you really doubt if this time was real or she was still just doing her job. My guess is that, Carrie Mathison was confused. Carrie has always been bipolar, this kind of thing would have been the perfect, subtle hint at how she would be if she didn’t take her meds.

Of course, there’s the reveal of Brody’s meeting with Walker’s rather sinister, but generally harmless, diplomatic contact. Even though I am watching it for the second time, I didn’t remember this scene and I was waiting with bated breath to see who was in the chair as the diplomat walks into his house. It was still exactly as jarring as watching it the first time, when Brody jumps out of the chair and lunges for the guy.

Another thing to look at and wonder about, was Mandy Patinkin’s acting and his character, Saul Berenson’s STYLE. The one interrogation scene that he has with Brody’s guard is enough to convince anyone of his prowess as an intelligence officer. Carrie, through all the seasons, has always used her eyes, her earnestness and oft-times her confused state of mind about whether or not she loves an asset / source to get intel. Saul used a standard set of negotiation tactics that he probably devised(?). The fact that the guard gives up the information that gets the professor is enough proof for his effectiveness.

Oh and: Saul and Carrie. Saul has a major major sweet spot for Carrie in the first season. She goes around him and through him several times just in the first 8 episodes. She begins with the bugging of Brody’s house, then she oversteps her boundaries at the Brody debriefing, then she goes and MEETS Brody despite knowing that that is the worst thing she could have done and she would irrevocably lose an unbiased boss after doing something like that. She keeps doing it and Saul keeps believing and forgiving her, hoping for something better.

The way they have come together again in Season 6 to work together has never happenned before. This time, she abides by the rules (except in that small Sekou Bah case where she overstepped, as usual), with Saul she is being a lot more careful. The indications for this are non-existent and I think they just come out of the writer’s psyche and don’t manifest themselves in season 6, but the treatment is clear.

The same set of writers has been writing and producing Homeland for the past 6 seasons, that has been one of the most important points in maintaining a consistent character profile for each character. Especially characters as complex as Carrie and Saul.

P.S. I spent (8 * 40) = 320 ~ 4.5-5 hours watching the 8 episodes. Atleast, I could write this post. I am going to do just this from now to create content.

POST #80 is OVER

Day 79 - Cruising Attitude - Heather Poole

2017-05-07T00:00:00+00:00

(May 9th, 2317 GMT+5.5)

I read Cruising Attitude today. I did that the whole day, I read about 53% of it. It is a book by Heather Poole about her foray into the Airline industry and talks about a lot of the intricacies of the flight attendant job, the training, the appearance that is almost always impeccable, the language, their body language, their walk through the terminal (!!).

The book featured in a recent NY Times article about what flight attendants learn about at de-escalation training, and at 260 pages, it seemed like a really good idea to start reading that book and finish it off as soon as I can. (I am running 3 books behind schedule on my Goodreads Reading challenge. I don’t see any holes in the next few months to be able to patch that up. PUSH NOW!)

The first half of it was really funny. It was also not coherent. It felt like amateurish writing with phrases repeated within the space of a 2 pages, which is more of an editing glitch, I think. But the fact that the writer is an amateur is not lost on me.

She has tried to tell a great story, which is commendable considering that it might have been just as easy for her to just write a few chapters and make each chapter about one particular incident and gloss over her own personal experience as the first chapter.

The concentration on her friends and fellow flight attendants till now has been indicative of the amount of sisterhood and comaradery that she feels about them. Including those she was at flight attendant training (which she prefers to call Barbie Bootcamp) with.

I hope to be able to finish off this book in the near future and move on to more ambitious reading projects. (This is certainly a day-off book with such easy reading and a bare bones, easy to follow plot) I started this book with the hope to really understand what flight attendants think and feel when they are on a flight and in charge of about 50 passengers each. That goal will be fulfilled by the end of this book, HEATHER POOLE, KUDOS!

POST #79 is OVER

Day 78 - Re-watching Homeland season 1; Claire Danes <3

2017-05-06T00:00:00+00:00

I have been rewatching Homeland episodes. Season 1, starting right from the pilot. I can’t get enough of Claire Danes and Mandy Patinkin before they became infinitely more matured and started handling actual, credible threats instead of gut feelings about someone who is off.

Those are the last few moments of Homeland 1.4: Semper I. Carrie just met Brody for the first time alone. Her smile says a lot here, she was struck by him right from the first time they met. I never noticed the subtlety with which these episodes were shot.

They way being a voyeur into Brody’s life made her feel bad, but she watched anyway because she still believed that even in those moments he might give something essential away. That quality stayed with her through 5 seasons. She watches Quinn’s painful video several hundred times, until she finds the tile design that eventually leads to them finding him.

It’s surprisingly how less their character traits have all changed, having a constant set of writers makes a huge difference in the continuity that a show maintains. Changing writers will always change the style of a show, and viewers will give up. Alex Gansa has done Homeland justice.

Do I think Homeland should end soon? Well, I don’t see any reason for them to end it. The story hasn’t gotten repetitive yet, they are still coming up with new plot points and insane insane twists that can’t be seen despite 5 seasons of material.

Did Quinn have a good exit from the show? HELL YEAH. I am sure Rupert will be back in past sequences in the next season, it’s a good old trick to keep the transition from Rupert to no-Rupert smooth. I am glad for that, but his character had become something that had only two goals, like one Reddit user aptly summarised it. “He has to get together with Carrie now, or die. I don’t see them killing him after this much groundwork that went into his character and building him to this point.” That served as a source of hope and inspiration, until the episode released. He had a good exit. Rupert Friend will return as another different character somewhere else, I am sure. I shall wait eagerly for that.

POST #78 is OVER

Day 77 - Next book that I am going to read, List of TV series to catch up on grows

2017-05-05T00:00:00+00:00

Cruising Attitude (Heather Poole) is my next read. It’s about a flight attendant’s view into making a flight work right. She had 15 years of experience before she penned down this book. This book also featured in a recent NYTimes article about conflict de-escalation, published in the aftermath of the United Airlines fiasco.

Several TV series are now running at the same time. Handmaid’s Tale, Silicon Valley Season 4, and Veep season 6. Lucifer’s Season 2 is resuming in two days! Prison Break’s season 5 started a few weeks ago, in a surprising turn of events, Michael turns out to be alive. Extremely strange! Sucre and T-Bag are still in the story, that should be a good watch.

I am listening now to Breaking Benjamin’s album Phobia now.

TODO: YOUTUBE INSERT

I spent all my material on Day 78 (which should have been after this post, but there was a slight ordering issue). It was about re-watching the first season of Homeland. I decided that I had re-watched enough Seinfeld (I have now watched almost every episode twice, and a few episodes upto 5-6 times, like The Stall, the one in which they get stuck in the parking lot and the episode in which they go to Hamptons to see the ugly baby)

POST #77 is OVER

Day 76 - I have nothing to write about

2017-05-04T00:00:00+00:00

I have been staring at an empty markdown file for the past 10 minutes. The difference between going to college and working a day job is that you are at your day job for 8 hours a day. Then, you come home to have dinner and chill. There’s no other time to read something or find something that is actually worth writing about. I must inevitably fall back on some of my reserve, can always write about topics.

Now, after 75 days of (almost) daily, (certainly) regular writing, that list of topics stands at [ ]: an empty list. My last post was about nothing, the post before that was about a book I read and quotes from that book, so I didn’t effectively do anything except read the book and read the parts I marked in the book again.

POST #76 is OVER

(Before it even began)

Day 75 - (Day 1 at Elanic) Books are being made into movies, left and right

2017-05-03T00:00:00+00:00

In the latest (April, actually) news, there are a few more book to screen adaptations that are scheduled to come out this year. Gillian Flynn’s final novel which hasn’t been adapted yet, Sharp Objects, is coming out as a series on HBO starring AMY ADAMS! After Arrival, she probably needs to step up her game to play the frontwoman.

More Stephen King with It and Dark Tower. It is HUGE, it’s about 1000 pages and I am not going to commit to that book anytime in the near future, which is reason enough to be excited for the adaptation. I don’t know how long Dark Tower is, but if it is anywhere more than 400 pages, I am going to skip it.

The Circle has been on my list for a while now, it should have been read before Emma actually. It’s also a considerably long book, and I am not sure whether now is the right time to start it. The adaptation is coming out very soon (or is already out). I could rush to finish reading the book and then see the adaptation in a theatre, the practicality of that plan isn’t something I can attest to.

Murder on the Orient Express goes onto my list, I don’t like reading Agatha Christie too much, but this one was really hyped up by everyone who read it. It might be well worth the hype.

Today was Day 1 of my 2 month internship at Elanic.

POST #75 is OVER

Day 74 - MISSED; DONE WITH EMMA (AUSTEN)! - Favourite Quotes (some)

2017-05-02T00:00:00+00:00

This is long overdue now. This post was supposed to be out two days ago.

I took 1 month and 5 days to read that book! It was 450 pages. THAT IS UNACCEPTABLE. Softening my stane a little bit though, considering how much stuff happened in the past two months (BTP Presentation, End term exams, looking for an internship) it isn’t too bad.

The book was GREAT. I get why Amy Dunne wanted everyone to read Austen. I am going to wing it for this post and put in some of my favourite quotes from Emma. This involves me (painfully) scrolling through my Kindle and reading the stuff I have highlighted. Especially in Volume 3, when the story was drawing to a climax there were a very high number of lines worth quoting right away.

Emma was aware that great must be the difference between a Mrs. Weston, only half a mile from them, and a Miss Taylor in the house; and with all her advantages, natural and domestic, she was now in great danger of suffering from intellectual solitude.

Miss Bates stood in the very worst predicament in the world for having much of the public favour; and she had no intellectual superiority to make atonement to herself, or frighten those who might hate her into outward respect. ***

real, honest, old-fashioned Boarding-school, where a reasonable quantity of accomplishments were sold at a reasonable price, and where girls might be sent to be out of the way, and scramble themselves into a little education, without any danger of coming back prodigies.

This one is especially worth noticing. Austen goes out of her way to rub this in. ***

What say you to Mr. Weston and Mr. Elton? Compare Mr. Martin with either of them. Compare their manner of carrying themselves; of walking; of speaking; of being silent. You must see the difference.”

This is representative of Emma’s outlook in the first volume of the book. She was a first rate snob.

And ever since she was twelve, Emma has been mistress of the house and of you all. In her mother she lost the only person able to cope with her. She inherits her mother’s talents, and must have been under subjection to her.”

Emma is really attached to her father, and this is definitely the only significant reference to her mother. This is in Volume 1 and after this, no mention of her mother anywhere throughout the book! Not even in memories, or otherwise.

Men of sense, whatever you may chuse to say, do not want silly wives. Men of family would not be very fond of connecting themselves with a girl of such obscurity—and most prudent men would be afraid of the inconvenience and disgrace they might be involved in, when the mystery of her parentage came to be revealed.

“My being charming, Harriet, is not quite enough to induce me to marry; I must find other people charming— one other person at least. And I am not only, not going to be married, at present, but have very little intention of ever marrying at all.”

“I have none of the usual inducements of women to marry. Were I to fall in love, indeed, it would be a different thing! but I never have been in love; it is not my way, or my nature; and I do not think I ever shall.

This is the defining Emma Volume 1 quote. It’s also extremely famous, and is quoted everywhere.

I shall not be a poor old maid; and it is poverty only which makes celibacy contemptible to a generous public!

This isn’t outdated, I feel.

things as he finds them, and makes enjoyment of them somehow or other, depending, I suspect, much more upon what is called society for his comforts, that is, upon the power of eating and drinking, and playing whist with his neighbours five times a week, than upon family affection, or any thing that home affords.”

She also more hated John Knightley, her brother in law.

it will be a small party, but where small parties are select, they are perhaps the most agreeable of any.

half an hour’s uninterrupted communication of all those little matters on which the daily happiness of private life depends, was one of the first gratifications of each.

Soon after undertaking this endeavour, I realised that it will take me an unreasonable amount of time to go through all my quotes and mark the favourite ones! This is the PDF of my quotes, generated by the Kindle. It has some profane language and all opinions that are truly mine.

POST #74 is OVER

Day 73 - Day 1 in Bangalore; Anticipation

2017-05-01T00:00:00+00:00

I am now in Bangalore. An interesting fact: The past three days, at 4 pm, I was in three different cities. 29th April: Kolkata, 30th April: Chennai and 1st May: Bangalore. I bet this is how Perry Wright (from Big Little Lies) feels.

There is a line in that book that stuck with me. There are several quotes that I still remember, because of the whole book + TV series thing, this one is definitely my favourite. They are speaking about Perry in general, and Celeste mentions how much he has to travel for work, and how he has to go around the world. And the other character replies, “Perry lives a rather exotic life, huh”. Travel suggests an exotic tinge to life, and that’s the way it has been, even though people who travel a lot for work are universally envied, their jobs are universally less appealing to everyone.

A very very happy turn of events is my progress with Emma! I am now done with 80% of the book! There’s still 90 pages of reading to be done, but I am feeling a lot better now, I plan to quickly complete reading this book and turn on to the next one. I have been stuck with this one book for long enough now. Not that I am complaining, but the commitment of huge books is not something that I am upto anymore. They take too long! (This was only 460 pages, compare that to the 5 ASOIAF giants at ~1000 pages each)

Handmaid’s Tale is out now, but on Hulu which doesn’t have service in India, sadly. Netflix and Amazon Prime Video are now in India as well, but the content segregation across these two services is too high. Half the comics have their specials on Amazon Prime. Vir Das now has a special on Netflix! There is no “over-arching” service which one might subscribe to and be assured to get all the latest entertainment. (I haven’t even mentioned HBO GO and Hotstar. HBO GO isn’t even available in India yet, but has almost all the HBO shows: Veep and Silicon Valley, both of which have an ongoing season right now)

Fragmentation across entertainment services is highly undesirable, I wish someone would fix this.

Forward looking P.S. Handmaid’s Tale’s Audio CD is on it’s way, I think. I am excited about that too. My laptop doesn’t have a disk drive, which would be a minor problem to get through before I can listen to the book

POST #73 is OVER

Day 72 - BHIM Aadhaar pay lacks a second factor; Elliot's paranoid nature

2017-04-30T00:00:00+00:00

Churning out two in a row, today. I am 3 books behind schedule, as of this writing. Today was perhaps the cut-off day for the third day!

Recently, I read this article about Aadhaar and how the core biometric database of a lot of Indians that is now held by UIDAI is secured, and will be handled in the future.

I have been reading a lot of material about Aadhaar, UID, UIDAI lately, and all of it has been pretty grim. With the introduction of things like BHIM Aadhaar Pay which requires the merchant to enter the Aadhaar number, and the customer to use their fingerprint for making a payment, it seems like Aadhaar will be a part of all of our lives, whether we like it or want it or not.

BHIM Aadhaar Pay is especially insecure because of the absence of a second factor. The only thing that is required is a fingerprint, and even The Verge wrote an article about this. It was pretty eye opening. Play Doh is a toy for kids, they make stuff with it, and now it’s being used once the fingerprint has been created as a fake finger? DERP.

This Guardian article has more information about the guy who got the German minister’s fingerprint using just her high-res photos.

Well, Elliot converted E-Corp to Evil Corp in his mind. Elliot was paranoid as hell and destroyed his whole setup even at the slight doubt of someone having figured him out. Elliot also mis-labelled his CDs as old rock albums to create an efficient smoke screen that worked rather well (because the people he hung out with weren’t into that kind of music). It pays to be paranoid, apparently.

POST #72 is OVER

Day 71 - A mad dash to the boarding gate; Where is Emma (the book) going?

2017-04-29T00:00:00+00:00

I am now in Chennai. 1 mad dash to the boarding gate later. The fact that boarding gates close 25 minutes before departure totally skipped my mind. I was strolling happily through security when Indigo ground staff called me on my phone and asked me to rush to the gate as the flight was going to leave!

I was the last person on the plane, and as soon as I walked in, they announced on the PA “Boarding Complete”. It was a slightly harrowing experience, but CCU is not a large enough airport and I am fairly familiar with it, so it wasn’t too harrowing.

I safely made the flight to Chennai, and in the past few months although I have come across several articles about a variety of subjects. One of the most eye catching was this New York Times article about the de-escalation training that crew members go through! It was such an eye opener that such training programs exist in the first place.

“My job is to create a situation where people can comply with me and still save face,” he said. “If they do, I won just because they got an explanation.”

There was also a reference to a book, Cruising Attitude by Heather Poole. At 262 pages, it’s fairly small and I can hope to read it. I have absolutely no clue when, or how, yet.

Emma (Jane Austen) is the book I have been struggling to finish ever since a long time now. I am 2 books behind schedule on my Goodreads 40 books this year challenge. Yesterday, being a travel day gave some good progress. It was still not enough. I am two volumes of the book in, and this is the third volume. Everyone has been sufficiently hated by Emma, everyone has sufficiently harassed Harriet, and adored F Churchill, and adored/hated Elton. Where is the story going and how is it going to come to an end?

I would bet that Emma ends up falling in love and eventually marrying one of the people. Frank seems to be the obvious choice, Knightley is interestingly in love with the idea of Emma but not with the person herself, it appears. Jane Fairfax is a crazy puzzle piece that fits somewhere. I am not sure where though.

All in all, it will be an exciting finish because seldom have books left the reader in such a dark alley, after making it 60% to their destination!

POST #71 is OVER

Day 70 - The existence of a Kurt Cobain biopic;

2017-04-28T00:00:00+00:00

This is definitely going to be unique. I am writing from an airport. Yeah, typing stuff on a laptop (even if it is gibberish or this) is an experience worth recording somewhere. I am in the Calcutta Airport, waiting to get on a flight to Chennai in about 90 minutes from now.

The shenanigans of gettting to the airport are hard enough, if you live in IIT Kharagpur or studied there. You get on a taxi from the institute to the KGP train station, a train to Howrah (one of Calcutta’s train stations), then get on a taxi from Howrah to the airport. There are so many legs you must go through to do a simple 2 hour flight.

Yesterday, 28th April 2017 was the last day of Semester 8. I definitely plan to write a semester review post, this is not going to be that. That has to be “written”.

In retrospect, after the United Airlines and (disproportionately) high number of incidents related to airports and air travel in the past 4 months, I am not really nervous by any accounts. Excited would sum it up more accurately, I think. Demented, some might complain.

Yesterday, I had a conversation about movies, Movies that are must watch movies and I was told about Montage of Heck (2015). It’s a Kurt Cobain biography movie thingy! KURT COBAIN <3 <3 <3. I have seen his interview clips (almost all of them) from the 1990s, and the persona of coolness that he exudes is incredible! YOU must fall in love with him.

Apparently, it’s a cartoon movie intercut with actual montages of his old photos! This is going right to the top of my watchlist for now.

POST #70 is OVER

Day 69 - Music and Temperature

2017-04-27T00:00:00+00:00

This is one of the posts that I hope to be proud of when I am done with this 100 days of writing marathon.

For a long time, moods and music have been closely connected. I think temperatures are too. Stay with me.

On a hot summer day (hot is about 35-40 degrees celsius, here in India), listening to an Arctic Monkeys song that is racing through is hard. I might listen to it for a bit, but I eventually end up feeling hot. Especially, songs from Favourite Worst Nightmare. The best example is If You Were There, Beware.

The songs starts out slightly less heavier than it gets, which makes it a bit more enticing.

Whereas, now compare it to the following two songs. They instantly make you feel light and cold.

All Lana Del Rey songs seem to induce a laid-back, let’s sleep this one out attitude on the listener. I first binged on Summertime Sadness for a whole month in the Summer of 2014. I hadn’t heard any of her other songs then, but that one song that I heard and it’s video was enough to put me through May 2014, in temperatures that were way over 40 degrees in Kharagpur. Listening to it now, I feel like that song was incredibly “hot” and there was definitely something off with me for having heard it and liked it in the summer.

Bonus fact: I had no clue what Summertime Sadness was, when I first heard the song. I liked the lyrics though, they were poetic and each sentence merged into the other. That is a feeling I have about Lana’s music even today. (She released a single a few days back.) Now that I know what it means, the video makes a lot more sense. It builds up to the climax that I didn’t understand before.

There are some songs I couldn’t quite figure out, though.

It’s probably because these songs don’t really have any unique “temperature” to them. They are cold-blooded, they adapt to the temperature their listener is in. That’s a pretty cool analogy, I feel. But, who am I, really?

P.S. (to a post that I was hoping would be great but turned out to be slightly above average): I have been meaning to do a chronological music catalogue in which I list what all artists, bands and people making music I love. Something that I could look back on 10 years from now and reflect at how mature / immature / misled I was, at this point in my life.

I would give anything to have a Year In Review video for every single year of my life. Something that is incredibly fast paced and basically a clone of this Casey Neistat video.

Day 68 - Internet shutdown in Kashmir again; Continous threat assessment

2017-04-26T00:00:00+00:00

Some more bizarre things happened today, 27th April, 2017. Not so bizarre, now that I know that Internet shutdowns were common. This is definitely the first internet shutdown that I have seen in the papers. Actually noticed. Kashmir now has a block on 21 social networks for the next month. These are the social networks:

I just had to laugh when I saw Tumblr, Pinterest and Snapchat on that list. Seriously, who is sharing even the anticipated objectionable materials.

More bad news for LastPass’ security team and all users of a password manager that is either Lastpass itself or something else. I have never been comfortable giving anyone access to my passwords. Even if it is a metal box that no human being promises to ever look at. Managing passwords on your own is slightly involved and often irritating, but it is totally worth the risk. The one thing that I would recommend that everyone do is: Keep assessing the amount of threat that you are under of being attacked, and the worst case scenarios if you indeed get attacked If you do this on a regular, unconscious level, then you will eventually not have to actively think about what might happen. You are always thinking about that.

A sample scenario: What happens if I left my computer unlocked for 10 minutes and went somewhere and someone used the computer for 9 minutes during that period. In that case, I would lose my RSA private keys and probably give them access to all the servers I have access to (and have stored in my ~/.ssh/config for about 6 minutes if they are really fast at getting situated) In 6 minutes, most servers can be taken down. They would still not have sudo access on these servers, so they would have to run something like the fork bomb which is easily solved by a reboot. The Metakgp server takes about 16 minutes to reboot, that’s a total down time of nearly 17-20 minutes at most. They would also get my RSA private keys, which would mean that I would have to contact someone to immediately remove my old keys and refresh them with new keys everywhere. This might take considerably more time and effort and this is probably the thing I would be most concerned about.

They might definitely be able to get a file-tree of my complete hard drive (although that’s pushing it, because the last time I ran tree / it took 1.5 hours to complete. Yes, that’s 1 TB worth of files’ list). They might be able to get some information, but it would still not be enough to do anything with.

But going into the browser, things escalate really quickly. If they don’t care about my RSA keys, then within minutes with a decent internet connection, the attacker should be easily able to cause havoc with my primary email account, a Facebook account. Send out emails which will be very hard to explain, messages that might be embarassing at the worst.

All in all, these threats are bad enough for me to force me to lock my computer before leaving, even if it is just for a couple minutes.

You might note that I have also just assesed the threats for the case where someone guesses or gets my computer’s login password from somewhere. The problems and the scenarios would be the same.

So, keep assessing and thinking about how things could go wrong. Most of the security stuff that I have realised have been in retrospect, and the helplessness is rather crippling.

POST #68 is OVER

Day 67 - MISSED; IIT Delhi's DNS provider got hacked, no post mortem though; Re watching Contagion (2011) and Another Earth (2011)

2017-04-25T00:00:00+00:00

Ah, I have been running two days late now. Right now is 27th April, 1733. This post should have been out on 25th April before I slept. I came down with a bad cold and fever yesterday. So, now I have to manufacture two posts in quick succession. I still have one more exam to give tomorrow. Nah, no big deal!

IIT Delhi’s website got attacked, alongwith a lot of other Indian websites. Now, apparently, it wasn’t the website servers themselves that got hacked but the DNS provider, ERNET that got hacked. And after the DNS providers got hacked, they remained in that state for about 12 hours.

Ironically, on the same day, only a few hours before this fiasco, this article about IIT Delhi using online voting systems for student elections came out. I don’t mean to get into an alma mater war here, but anyone who has designed and built web applications knows that voting systems are rather simple systems to build because the constraints that are applied in the real world are pretty stringent: fixed number of voting booths, constant surveillance of the people coming and voting, validation is done by human beings and not the software. Also, IIT Kharagpur has been using online voting systems for the past 4 years at the very least. Probably, even before that.

The unfortunate part of this is that there won’t be any post mortem of this particular attack. Cloudfare recently got attacked with their weird missing NULL character bug, but they published a detailed post mortem about the attack, when they found out about it, how they mitigated it and all kinds of details. That kind of data reassures users and also helps others prevent similar attacks against them.

I also ended up re-watching two movies that I have watched several times already. I don’t know what I hope to get from watching them again and again, I believe there’s something about the story, the sophistication of the story telling, the subtlety with which the story unfolds and most importantly, the hope that is the underlying driving factor for both stories: Contagion (2011) and Another Earth (2011). Surprisingly, both movies were made in 2011.

On an auxilliary note, Audible got back regarding the main narration on Handmaid’s Tale’s audiobook.

Just a few added scenes. Their customer service response was rather fast, to be honest.

POST #67 is OVER

Day 66 - Examinations and Time

2017-04-24T00:00:00+00:00

At school and college, there is a laid back approach to exams. 5 exams in a month, in some places. Some study leave before the first exam: in the order of a week to a month for practical exams. Lots and lots of free time. Before the exams begin, before each exam. All of this is wasted time.

Not here. Both the mid term and the end term exam sessions are 7-8 working days. They generally start on a Monday and end the next Tuesday. Sometimes, they start on a Thursday and end the next Friday. There’s no official “study leave”. Professors start to wind up their teaching about a week before the last class if they have planned it well, a lot of professors teach right till 2 days before their subject’s End Term. That’s just how it works. Exams are a marathon Inspite of that, I am glad for it. All of the time that is forced on you to study for an exam, is wasted time that you could have spent doing something more important and worthwhile if there wasn’t an exam looming over your head.

I woke up at 7 am this morning, I have an exam at 2 pm. I am not frantically turning pages and studying. It is 9:45 am right now, and I haven’t opened a single notebook yet. I have stolen away this time from my preparation to do something I love to do, knowing that the time I have after completing this post will be enough for me to get through whatever preparation is left.

The exam will end at 5 pm, my next exam is on Thursday. There again I will steal an hour’s worth of doing something I like: Reading Emma, writing some more, working on a project.

I write about this today because when I woke up this morning, I realised that I had woken up early and I wasn’t feeling bad about it. I didn’t have to be anywhere, yet I was up early. If anything, I was happy about the few extra hours I was getting. This essay by Andrej Karpathy is on point. My favourite quote from it is this section, right at the end of the essay:

Undergrads tend to have tunnel vision about their classes. They want to get good grades, etc. The crucial fact to realize is that noone will care about your grades, unless they are bad. For example, I always used to say that the smartest student will get 85% in all of his courses. This way, you end up with somewhere around 4.0 score, but you did not over-study, and you did not under-study.

Your time is a precious, limited resource. Get to a point where you don’t screw up on a test and then switch your attention to much more important endeavors.

I have never been asked about my grade point or my grade in a particular subject during an interview. I feel rather proud about a few and used to put them in my CV, until I realised that no one cares. As long as I have a number above 8 out of 10 in my CV, they know that I haven’t screwed up really bad and hence, it isn’t even talked about in most interviews.

The number of exams that I have given from being a staunch part of the Indian education system is staggering. I have been giving exams right since I was 6 years old. School was filled with periodic exams. Preparation for JEE 2013 increased that number astronomically, with 6 hour tests given every other week for a whole year. Just after coming to college, I have probably given hundreds of exams and class tests and lab tests. All of this test taking makes you a little bit de-sensitized to an exam. They become regular occurences that hold no more significance than you choose to give them.

I would love to write a TL; DR for this post, I am having a hard time figuring out what it A would be though. I wanted to make a point about exams.

Exams are not about how well you are prepared for them or how much time you spent preparing. They are about recognising what you need to study and where that will get you. They are about optimizing your preparation so you can get the best out of each test. They are about understanding the circumstances of every single course: the professor, the grading patterns, the question papers. They are about figuring out what score you are going to be okay with, because that number is different for each one of us. Lastly, they are about realising that when exams are looming over your head, you needn’t hit pause on the rest of your life. You can plan well enough to keep your life playing and still score well. It’s possible, and once you give a lot of tests, you will end up doing it subconciously, there will be no explicit writing down of dates and times.

That’s a good TL; DR, I feel. It encapsulates what I have always meant to tell students giving their first few exams. Exams are a complicated topic of discourse because of all the uncertainties and variables that come with every test you take. I have tried to summarise my experiences above. If you have given a lot of exams and have wanted to share your perspective with the world, sure do. The more viewpoints new people get, the more easily they can find their own.

POST #66 is OVER

Day 65 - Handmaid's Tale performed by Claire Danes, Fargo (1996) - DISTURBING

2017-04-23T00:00:00+00:00

Handmaid’s Tale was released as an audiobook. That’s not the best part of the news. The best part: It is narrated by CLAIRE DANES. OH MY!

I found the above excerpts on Audible’s Soundcloud, and they are DOPE!. This audiobook also has some new material, that was added by Margaret Atwood herself!

The TV series of this book is going to premiere on 26th April, in 3 days. The trailer for that series (which is going to be on Hulu) is also stirring. There’s a lot of new material for this book coming out this year. AH, SUCH GREAT TIMES!

I was rather stunned with the Audiobook version that Claire Danes performed. I have since found that there are two versions here, in 2013, Claire Danes performed this book for the first time. THat won a few awards and was wildly popular.

Now, it seems she has performed this again(?) or they have used the same audio again. From the reviews on the new Audible listing, it seems that they added scenes to the audiobook, an essay, an afterword by the author herself, a Q&A with the professor (which appears as an epilogue in the book), but the main voice acting that Claire Danes did has been left unchanged. The old Amazon.in listing for the MP3 CD that has the older audiobook is great! I think I will pick that up.

Audible’s pricing is pretty confusing too, I am not sure if it’s a subscription or if you can simply buy a single book and be done with it. Why does it have to be so confusing? Amazon seems to have made buying Books and e-Books simpler, but Audible is still a confusing maze that I have been stuck in. (I would love to use the trial and buy this book, but would it then mean that I will have this book forever or it will just lapse after 30 days and I won’t have access anymore?)

There’s also this YouTube promotional video that was released in 2012:

The quiver in her voice when she talks about “Her Story” is very recognizable from her difficult scenes in Homeland (especially the one in which Quinn ( :( ) and Carrie shoot a video to show Franny in the previous season).

I watched Fargo (1996) today. The fact that these events actually happened in reality is pretty disturbing.

POST #65 is OVER

Day 64 - My thoughts on a Quora answer I disliked, why are titles on Medium so clickbait-y?

2017-04-22T00:00:00+00:00

Yesterday, a friend shared [this Quora answer]https://www.quora.com/What-are-some-psychological-facts-that-people-dont-know-1/answer/Dorottya-T%C3%B3th) which ended up with it appearing on my news feed.

I read the answer and instantly disliked the contentions made by the author. I was going to let it go, until someone asked me to ellaborate on why I disliked the answer. I had to think about this for just a bit, after which I came up with this extremely long answer:

Ah, thanks for making me think more about it! :D

First of all, if writing was Tom’s true dream, he ought to have done something concrete towards it. But let’s not talk about that, because he probably has a lot of reasons why he can’t do it.

Why Sam doesn’t exist:

If writing is your dream and you have a day-job, you can find maybe 1 hour of worthwhile free time that you might spend doing something else. Say Tom doesn’t tell anyone about his writing habits, and just does it alone (apparently Tom can achieve anything then), does he write enough?

When you start anything big, the excitement is very high, you might put in upto 6-7 hours of writing the first day, but you will always get to a point where that initial excitement is gone and this is the pivotal point for any project. You must either struggle through it and stick with your project or ditch it. This is when you struggle for motivation, struggle to find content, struggle to decide whether you should keep at it or just drop it.

If you made a public proclamation, your friends will keep asking about it, the closest ones for sure. It’s embarassing to accept that you didn’t go through with something that you were so passionate about, because accepting that you stopped to someone else is simply an indirect way of admitting it to yourself. You really wanted to do something, and you couldn’t go through with it. What does that say about you? How did your perception of yourself change? Negatively, I argue.

Someone who doesn’t tell any of his friends anything and just publishes a book doesn’t exist. That just doesn’t happen. Everyone slowly moves towards their goal, they don’t just GET there.

Why this is a generalisation:

Everyone is different, following your dreams and passions is a topic that’s been heavily written about offline and online. People have always wanted that single formula to get there, to become a writer or a film-maker or whatever. The fact that it’s been so long, and there’s no formula yet is enough to prove that no such formula exists.

No such formula exists because EVERY SINGLE SITUATION IS DIFFERENT. That’s the core reason for my dislike of this answer. This is an overly simplified answer about a very complicated, specific problem: How do you get to where you want to be, from where you are? There’s no one single way to do that, and consequently, there’s no single way you can completely screw it up either. Every step in any direction is a step towards the goal or away from it, but making progress is more important than making sure that each step you take goes towards the goal. (Some of Casey Neistat’s videos have really detailed explanations of goals and life and the path to take to get there. All of the above has formed from my own experiences with ambitious projects and accomplished people’s views on the topic)

Nit picks:

The question asks for psychological facts, facts are generally supported by research that is either biological or done using control groups and analysis. They are not things that you can come up with using fictional characters and thought experiments.

The comments of that answer have some references to a 2009 paper. That paper is shaky, IMHO. If you believe that the experiment that was performed was to the point and is an accurate representation, good for you. :)

Relatability:

There’s no getting around the fact that more than 10,000 people thought that answer was a good one. Maybe the answer is relatable, that doesn’t make the answer true. Just because something happened to 10,000 people doesn’t mean that’s the only way to do that thing. There are other ways too. :)

I have definitely written down everything that I didn’t like about the answer. One of the responses to this comment was very interesting:

I agree to your point that it’s an “overly simplified answer” - something most writers on Quora/Medium tend to do to bring the focus on to one focal point, often off reality. For example, I completely agree that the ideal Sam as described doesn’t exist. It’s sometimes got to be taken with a pinch of salt.

(emphasis mine)

The bringing the focus onto one point is a major reason for my moving off of these two platforms. Medium especially faces this problem a lot. Most of the articles on Medium are preachy, clickbaity or give you a list to do something. Frankly, it’s rather irritating.

While Quora is something that has content which can be enjoyed as entertainment, one must never make the mistake of taking Quora answers and associating them to words that are even near as strong as “facts”. Most things on Quora are experience based, and only those questions are worth reading. That said, there is something off about these two platforms. Maybe it’s the lack of curation and the algorithmic suggestion of articles based on our past reads, which in most cases are just one-time reads to understand a topic.

If I saw a Medium article about “Why people should write”, that doesn’t mean you fill my newsfeed with “How to become a great writer in 6 months” articles for the next 10 years. That doesn’t make any sense. These platforms perhaps work best when you use them as your primary source for user created content. If you consume content across the Internet, you are probably going to have a bad experience with them.

To prove my point, I will show the first few posts on my Medium feed:

** ** ** ** ***

Now, I am aware that I am judging an article by it’s title and thumbnail and that some of these articles might be really good. But the fact that all those titles irritate me and I would never click on something like that is enough to understand that I would never get to know whether those articles were good or not. Can we tone it down, please?

I would like less Clickbait-y, more news-like titles which don’t give me a list, or tell me what I must do, but a title that tells me what the content of the article is.

Now that you have reached here, take all of that with a table spoon of salt because I don’t spend any time on Medium and I don’t know anything about how the articles with irritating titles are underneath the hood.

POST #64 is OVER

Day 63 - Empty post

2017-04-21T00:00:00+00:00

This is going to be a rather short one. There was a suicide in IIT KGP today. Everything seems trivial for now, it’s become a HORRIBLE DAY and it’s a horrible thing to happen to someone.

Day 62 - Frustratingly stalled at 40% of Emma, Juicero meltdown, Plastc shutdown despite $9 million

2017-04-20T00:00:00+00:00

Emma has been out of this blog for a few posts now. I have made absolutely no progress with it! Frustratingly, I am still stuck at around 38%. With exams Saturday, and 4 days of the next week, there is absolutely no way to read any more.

There was no resisting such news, no possibility of avoiding the influence of such a happy face as Mr. Weston’s, confirmed as it all was by the words and the countenance of his wife, fewer and quieter, but not less to the purpose.

– Mr. Weston just told Emma that Frank was going to visit the next day, at dinner time.

Okay, I read till about 40% in the past 15 minutes. This was the first time that Frank meets Emma, I have a feeling I know how this is going to go. So, basically, Emma loves Frank, but Frank loves Jane Fairfax(?) or Harriet. Harriet is definitely a stretch, because Frank appears to be the cunning wolf-kinda guy. All of this complete speculation, I am rather confident in it though.

Juicero had a meltdown, sort of. Bloomberg published a video that said the juice can be squeezed by hand, and the machine wasn’t needed, and Juicero’s CEO immediately shot back on Medium and offered to refund everyone who bought their Juicer machine.

That means that if you send us your Press, we’ll refund the money you paid for it. Period.

For starters, that’s a really really good save. While how many people might actually refund their machines, is still to be seen, I think most people will probably not do it. The convenience that a $400(!!!) juicepress machine offers is much much higher than the hand squeezed juice that could be made out of the same packets.

I wish Uber had execs like these who would come out right after something bad happens, talk about their product, apologise, accept that they need to improve and TAKE ACTION! Unfortunately, they don’t.

And while that was happening, a plastic card company that promised a single card that could store a lot of cards closed down. Amid really unusual circumstances, they had about $9 million in pre-orders and couldn’t close two rounds of about $8 million dollars in funding. How could they possibly need $17 million to send out pre-orders? It’s very unusual.

Where have we seen this before? One word: Coin.

Coin was acquired by FitBit and there has been no word since, they ran their service for some time, and after that they shut down everything. What is it about these digital cards and companies that make them not making it?

POST #62 is OVER

Day 61 - So, you think webcams can't be turned on without the indicator light?

2017-04-19T00:00:00+00:00

2 days ago, I wrote this list about improving your security online and practices that you should follow. The last point on that list was Tape your webcam. The question that most people seem to have is:

Are there any existing exploits which can turn on your webcam without turning on the light that is there next to it? How widespread are these lights?
Why should I tape the webcam if I have nothing to hide? (This was the argument used by Shailene Woodley’s character in the movie Snowden (2016)).

I think the FBI director Comey answered both these the best:

I saw something in the news, so I copied it. I put a piece of tape — I have obviously a laptop, personal laptop — I put a piece of tape over the camera. Because I saw somebody smarter than I am had a piece of tape over their camera.

– From npr.org

And what did he say about the second question?

You go into any government office, we all have our little camera things that sit on top of the screen. They all have a little lid that closes down on them, You do that so that people who don’t have authority don’t look at you. I think that’s a good thing.

– From hindustantimes.com

The existence of exploits might be questioned now, but by the time that these exploites are released to the public, it will most certainly be too late to tape your webcam.

Moreover, if you know about the Lower Merion School District case in which the School district gave laptops to all their students and installed an anti-theft tracking software and used that to capture photos of their students at periodic intervals. Most students noticed that the green light next to their camera used to flash now and then, and they found it creepy but the people at the school reassured them that it was a technical glitch (which is the easiest out) and let it go. They paid heavily for their indiscretions and SICK actions, it must be noted though that the photos are out there and they might have been deleted or handed over, nonetheless some people have seen it.

Further research for about 30-40 minutes on sites like the StackExchange site for Information Security, Quora and news websites led me to several articles that detailed exploits and attackers getting access to webcams and using those to click pictures. There is a dearth of articles in 2014 and 2015, which is intriguing, but 2013 was the time that this issue was at it’s highest popularity and there are many articles from the second part of that year. So, read on!

September 2011, August 2016, Discussion on this InfoSecurity Stack Exchange question about whether webcams can be turned on without the indicator light
December 2013, CS professor and student from Johns Hopkins University publish a paper about turning on webcams on Macbooks without turning on the light
December 2013, Errata Security found an exploit to turn on webcams without the indicator light using DLL files
July 2015, A camera connected to the internet was hacked, through the router and eerie music / messages were played. In what was perhaps a prank by some hacker, this is one of the most disturbing pieces on this topic
August 2015, Couple watching Netflix was inadvertently caught on camera by someone who hacked their computer’s webcam
February 2016, Edward Snowden says that NSA and GCHQ have been surveilling laptops of innocent people since 2008
April 2016, FBI Director James Comey says in an address about privacy and encryption that he tapes his webcam, and everyone in government offices does, and according to him, that’s a good thing
June 2016, Mark Zuckerberg tapes his camera and mic
August 2016, An answer on SuperUser Stack Exchange about how to turn on the webcam without turning on the light on Windows and Linux
April 2017, A Quora question with some surprisingly good answers about whether taping your webcam is something everyone should do

By this time, after reading all of that, I think you should be sufficiently convinced about 1. the possibility of the existence of exploits 2. that someone or the other is watching and it’s not right to let them watch without your permission.

On a sidenote, I didn’t find those articles in the exact chronological order that I have presented them in (obviously). After creating a list entry for each of them, I had to sort them. I probably did using Quick Sort, because I inherently created year pivots and grouped all of 2016’s articles together, same for 2015, 2013 and then once I had a clear distinction between these two sets, I moved the whole 2013 before 2015 and so on. Probably not traditional quick sort but I think that’s the algorithm that comes closest to describing what I did.

POST #61 is OVER

Day 60 - Nostalgia hits - Masaan and Lunchbox, Search for the Indian Railway Commercial Ticketing Manual

2017-04-18T00:00:00+00:00

Yesterday’s post, that I wrote 13 hours ago, was one of the most detailed and informative blog posts that I have written in this series for about 20 days now. I shared it with a lot of people, I hope people see it, read it and implement some of those measures in their lives too. For me, to implement all of those things took almost a year. I started with 2FA and now, I have implemented all of them. It takes considerable time and effort to implement some of those measures, but the time spent will be well worth it.

I had a strong bout of nostalgia today, movies related. I watched scenes from Masaan (2015) and Lunchbox (2013). A few letters, some great songs, some incredibly good actors. And scenes that are memorable for life. I wish they made more movies like this in the present times. Unsurprisingly, when I think of the last movie that I saw and really liked, the latest one I can remember is Yeh Jawani Hain Deewani which was released 4 years ago. It’s a sad state of affairs. (P.S. I really wish the publicity for Masaan had been great and I had seen that movie in a movie theater!)

Masaan had an especially dark middle part of the movie, it was a heart breaking scene to be honest. There are many spoilers to be given here, and I won’t talk about them. It hits you like a train would, I imagine.

Lunchbox had this insanely suspenseful sequence in it’s middle when suicide is on the cards for Nimrat Kaur’s character and her daughter. The picturing of the whole thing, my knowledge of the copycat effect and how it affects events like these, had my brain running full speed to find some way out of this. Some way to convince myself it won’t happen. It didn’t end up happening, I am SO glad for that. Sad endings only work to give the movie a bittersweet taste where you remember the movie for that particular bad event and everything else happened before and after, but that one bad event is always the defining moment for you in the movie.

In Masaan, Pankaj Tripathi’s character, who is an Indian Railways ticketing agent, has a book called the “Indian Railways Commercial Ticketing Manual” in his hand. Apparently, this manual consists of the instructions for new ticketing agents to learn how to generate tickets for their customers at the counter. A preliminary search lead me to this listing page for Codes, Manuals related to IR. The only manual that I could find which had a similar name was the Indian Railways Commercial Manual: Vol 1 I can’t be sure if this is the same book or not, I see no reason for them to put the manual online for anyone to read, but the difference in name could simply be because the online document was last published in 1992. That’s right! 25 YEARS AGO! There must definitely be a newer, better document that has the exact Ticketing information.

Considering that I wrote a post about this, it is my duty to find out as much as I can. Expect a comprehensive account of my search for the evasive Ticketing Manual: Vol 1 in the coming few posts in this series!

POST #60 is OVER

Day 59 - A list of things to improve your security online

2017-04-17T00:00:00+00:00

A new day, a new attack on HTTPS.

Can you tell which is phishing?

You can't. That's the problem with Unicode look-a-like character substitution. https://t.co/431d4dNwGT pic.twitter.com/r18CLNCXcE
— Michael Coates (@_mwc) April 15, 2017

Checking certificates is a great practice, and almost always I stop after opening a certificate and looking at the certificate signing chain. That tells most of the story. Specifically, when using some other computer, it tells you whether the computer’s owner is trying to fake some certificate sign by putting a malicious Gmail certificate on the computer and effectively implementing a MITM attack. It’s as simple as installing MITM Proxy which automatically creates a certificate and adds it to /etc/ssl/certs.

The best way to avoid this is to not login on any unknown device, at all. There are some guidelines like these that I have learned from experience, and although listing them all down is going to be next to impossible, I can try.

DO use Google Authenticator for 2-Factor Authentication. Use it for all the accounts that tell the world who you are: Email, Twitter (THEY IMPLEMENTED IT!), LastPass, GitHub, Digital Ocean, Slack. (Those are a few of the services I use 2FA for)
DON’T login to your core accounts on an unknown computer. There could be anything ranging from a hardware keylogger that you didn’t notice, a software keylogger which can be installed in 2 minutes on Windows or a malicious SSL MITM attacker who went to the pain of faking a certificate chain to reduce your suspicions. JUST DON’T. To download or share files, read the next point.
DO setup a publicly viewable Google Drive folder and create a shortlink for that folder and add it to your contacts. Whenever you want to download something on a strange computer, use your phone to put it on the public folder, download it and then delete it from Drive.
DO use DuckDuckGo. They don’t track you, they don’t even ask you to sign in. If you must use Google because sometimes DDG just doesn’t cut it, open Incognito and use !google before your query. That way the searches are not tied to your account and you can save yourself from targetted advertising based on your searches.
DO use a VPN whenever using Public WiFi. Something as simple as Surfeasy is enough to thwart people trying to spy on you using simple things like Wireshark. I know hardly anything about the myriad list of tools available out there to spy on people who are using a public network. Using a VPN should preclude some really basic attacks.
DON’T enter your Bank Password or anything related to Internet Banking WITHOUT first checking the SSL certificate of the website. If chrome shows the green bar, don’t stop there. Press CTRL+SHIFT+I, open the certificate and atleast look at the certificate. Look at the spelling of the website (Our brain does this weird thing where https://onlinsbi.com looks so so similar to https://onlinesbi.com. I used to be terrified of this particular attack.
DO use STRONG PASSWORDS. This is a no-brainer, really, but I still feel like I must say this. 8 character long passwords are the bare minimum and just don’t cut it. Something as simple as thc-hydra has bruteforcing options for passwords that are this small.
DO use LONG RSA keys. If possible, use Elliptic Curve keys. In any case, your default RSA key should ideally be 4096 bits long. This guide will help you upgrade your keys
DO have a small Pendrive that has your RSA keys, PGP keys (Who uses PGP anymore?), your 2FA recovery codes. Take it with you whenever you are going on a sufficiently long trip. Remember to use LUKS and encrypt the partition that has all this data. Otherwise, you are just inviting trouble.
DO uninstall openssh-server from your main computer. It’s an annoying package that doesn’t come pre-installed in Ubuntu, but if you ended up installing it somehow, uninstall it right now! sshd is one of those tunnels into your machine you didn’t even know existed until someone walked right in and stole files from you.
DO follow Security and Cryptography blogs using something like Feedly. The best two out there right now are cryptographyengineering.com and Bruce Schneier on Security. If you use Twitter a lot, follow some Security driven accounts like Filippo, @SwiftOnSecurity, and @evacide. I know I follow a lot more people who are in the industry, but I can’t seem to remember them. I would like to create a Twitter list of these people and then, pass that list around. (That list probably already exists?)
DO follow Cryptography news closely and go beyond the mainstream media to find out exactly what the attack is. Media headlines are generally click-baity and say things like The NSA can now see your bank passwords when the actual attack is an attack on HTTPS which requires a particular set of conditions to be met, and these conditions are never really met for most people. This is just a random example. My point is: Go beyond the mainstream media, go to the paper / article that the attacker published, read through it, think about it, and finally, take measures to preclude it.
DO use Full Disk Encryption on your Ubuntu main computer. It’s a pain to setup, you probably are going to have to take a whole backup and then encrypt and copy back to your disk. I did it when I re-installed Ubuntu. Do it right away, it is well worth it. If you don’t do it right now, definitely do it the next time you upgrade your operating system.
DO tape your webcam.

That list is no way complete. I am sure I must have missed some major point that I really wanted to tell people about, but didn’t include in this list. I won’t be editing this list though, I will probably publish a better version of this list in a couple months, if I have something more to say.

The bottomline is: do your best to close all the doors and windows and air-conditioning ducts that enter your online apartment. If you leave doors open, people will try to get in. If you close most of the doors, they will be discouraged, and (hopefully) leave you alone and target someone who is more vulnerable. And if you find that a door has been inadvertently left open, TAKE ACTION! Inaction now and regret later are a toxic combination, and the helplessness when you realise that you could have and should have taken steps after you have been attacked doesn’t help. It’s a bad bad feeling.

POST #59 is OVER

P.S. I am pretty sure this is one of the most coherent posts in this series. I have been writing this post for the past 45 minutes.

Day 58 - Reddit karma now at 978, animals see themselves in a mirror and Gnome's weird semantic versioning conventions

2017-04-16T00:00:00+00:00

Today my Reddit comment karma increased a bit more. It actually moved slowly towards 1000. It’s 978 right now. This particular comment got me 388 KARMA! WHAT?! STAGGERING.

Anyway, I also read an article about how animals might recognise themselves in the mirror. The tests are admittably absurd and although they appear legit, I feel like the opposition to it have a very very valid point. Nonetheless, it’s an interesting article and you should definitely give it a read.

Nautilus is, in general, a great publication. I subscribed to their mailing list which sends out emails every day (or more frequently?) with links to atleast 4-5 articles and I am never able to complete reading them by the end of the day. That eventually ends up in a lot of unread email that never gets read and eventually gets archived. I wish there was a simpler way to make a note of articles that I really want to read (by looking at the headline and the first few paragraphs of the article).

The other Nautilus (File Manager on Ubuntu) that I ranted about yesterday became rather anti-climactic, when I posted the same thing on r/ubuntu and u/TingPing promptly informed me that I didn’t have the latest version of Nautilus installed.

Gnome 3 uses a weird semantic versioning convention system that leads to 3.14 being the 7th major release (because only even numbers are major releases) and the latest stable release is 3.24. Ubuntu apparently ships by default and only allows the older release to be installed through their software repositories. The only way for me to fix this right now is to download the source and build Nautilus for myself. If the partition irritation is solved, then the ordeal would have been well worth it.

Initial build failed because of the absence of a package called gio2. I hope to pick this up some time in the near future and see if building solves the problem.

ONWARDS and UPWARDS! (George says this in a Seinfeld episode)

POST #58 is OVER

Day 57 - Double-Entry Book-keeping; Homeland Season 6 Finale - Let's talk

2017-04-15T00:00:00+00:00

In yesterday’s post (which I posted about an hour ago), I said that I was going to figure out how GnuCash worked. I made short work of that, and I have ended up learning what the basics of Double Entry Bookkeeping are:

The whole software makes a lot of sense now. I have setup the initial entry to get started with the expense stuff (I like the report feature, hopefully I should be able to generate some meaningful reports in a few months).

I think it’s finally time to talk about the Homeland Season 6 finale. Yeah, I should get this out of the way.

MAJOR SPOILERS AHEAD; DON’T CONTINUE IF YOU EVER PLAN TO WATCH HOMELAND

There’s a reason that spoiler notice is in HUGE font, I have often seen people disregard that and I have been tempted to disregard it often. The couple times that I did do it, I tried really really hard to forget the spoiler, which only ended up solidifying the spoiler in my head. (Eventually, I ended up not watching the thing and waiting for about 6 months or something and then coming back to watch it, that way the spoiler even if remembered comes to my mind only at a much later stage)

So, Quinn dies. Saul lives. Keane lives and when Dar Adal says “There’s something off about her”, he was SO RIGHT. Keane has basically started her war against the intelligence community. The stage for this whole thing starts with a plot to assasinate Keane that is foiled by Carrie’s weird intuition and a phone call from guess who? DAR ADAL. This one dialogue stuck in my head:

Carrie: Quinn? Are you seeing this they are evacuating the hotel. Quinn? Dar: This isn’t Quinn, Carrie. What’s going on? Carrie: Fuck you, Dar Dar: Listen to me, and listen very carefully. The President-elect’s life is in danger … Under no circumstances, allow her motorcade to leave the hotel. Do you hear me? Carrie: Why not? Dar: Because that hotel has been vetted for months. Swept thoroughly and repeatedly. The bomb-scare is a ruse to get her out on the street

Eventually, Carrie does stop the motorcade and Keane is saved. Quinn has to lay down his life to save Keane. Quinn dies. It was a SAD SAD MOMENT. Rupert Friend played one of the best characters in this show, from the first scene that Quinn walks in on Carrie during a mission (Dar sends him) and Carrie thinks he’s there spying for Dar and he helps her out. From then till his last scene, not once did he let Carrie down. The shootout where there are three people in the car with the mole inside the embassy, Quinn comes through for her. Carrie then stays back and searches for Quinn and figures out where he is.

TL; DR Quinn was my favourite character, has been for the past two seasons. Carrie is the lead character, so her being a favourite is no surprise. Quinn though. Oh, QUINN.

After a six week jump, Carrie is still working for the president and is some kind of a liason between the President and the intelligence community. O’Keefe still continues his rhetoric (which is infuriating but you know he’s not going to change and he doesn’t do much damage, so let him rant). Max and Carrie cry in the heartbreaking scene where they look at the photos that Quinn has. There’s a smiling picture of Carrie in them along with his son’s photos.

I wish they would give him some sort of a prequel episode where he is with the mother of his son and how they came about to be together and what he did that separated them. I wish they would bring Rupert Friend back, even if it is just for a cameo in one episode in season 7. I would say there’s a non-trivial chance of them doing it because Quinn has quickly risen to become one of the most beloved characters across the Internet. (Not to mention #EmmyForRupert)

All in all, this season has been one of the best TV show renewals. They have definitely recreated the magic, suspense, the thriller in the plot that was there in Seasons 1 and 2. At the same time, they have managed to provide the PERFECT setup for the next season with Keane going all rogue and imprisoning Saul (Dar was already in prison). HELL YEAH!

POST #57 is OVER

Day 56 - MISSED; Nautilus update is irritating, switching to Dolphin(?)

2017-04-14T00:00:00+00:00

So, I recently updated to Ubuntu 16.04. (Not really recently, I updated 41 days ago :O )

Alongwith the update of the distribution update, I got an update for most of the native apps. One of them: NAUTILUS. That’s the file manager thing on Ubuntu. It’s standard, it’s everywhere, I loved it. Used to love it. Now, they have done something annoying.

Most people have many partitions, I have several of them. Some of them are encrypted, some of them are NTFS to transfer files between Windows and Ubuntu, etc etc. Earlier, there was just one icon on the left-side task bar. Clicking once and opening it anywhere would add a window there. And now, things are different.

Each partition opens in a new window and all nautilus windows are not grouped together. Instead, partition windows are grouped together. This is INCREDIBLY inconvenient. I don’t have any partition up top because that takes up a lot of icons, and that’s not what I want to waste space on. So they are dumped right at the bottom of the sidebar, and it’s so HARD to navigate and copy files between partitions. I have absolutely not clue why or when they decided to go ahead and do this.

The problem is so bad that even if you open a new tab in a window and then open a new partition in that, then the tab appears as a window with some weird stuff happening. (Like if I have partition 1 open in the first tab and partition 2 in the second tab, clicking on the partition 2 brings that window into focus although visibly there’s only one window!)

It seems that they have simply removed having a Nautilus window unless it’s in the Home partition. Everything else is another window. It’s so frustrating, I am going to be going out there and looking for a saner way to manage my files. (I am installing Dolphin right now, hopefully Dolphin manages this situation better)

I know that I promised to write about Homeland’s season 6 finale, but this particular rant took precendence over that. I have been silently enduring this for long enough, I figured.

Another interesting application I just installed was GnuCash. I use terminal-wallet for my day-to-day finances and an Excel sheet for a larger picture. GnuCash looks like the right replacement for the larger picture one, it has a sweet interface that I already like, but I haven’t tested it out well, and the number and type of accounts is confusing. I am not surprised by that though, I will crack it.

P.S. Today was also the day of an Open Mic called Live at the Apollo 13, there were 6 acts and it was GOOD! I missed the first time that this happened (last year) for some mundane reason, Yesterday was GOOD!

POST #56 is OVER

Day 55 - BTP final presentation; Metakgp Talks session 2; Watching random Seinfeld episodes

2017-04-13T00:00:00+00:00

Today was the day of the BTP presentation! It was FAR BETTER than my expectations, and the uncertainty has all been over-written for now. In about 2 weeks, I will leave for Chennai and subsequently from there to Bangalore. In those two weeks, I have to give 5 end-term exams for the 5 subjects that I took, most probably give a lab test for the Operations Research Lab.

Meanwhile, I was thinking about a Semester Review post on the 30th of April, which is the first non semester day or something like that in which I go over what all I think I learnt this semester and is worth mentioning. The more I thought about it, the more sense it made, it would be like a real timeline of the things that were done over the past few years.

This being my 8th semester, I have missed the oppurtunity to right such posts for my 7 previous semesters, I would have loved to have them for each semester! Writing them in retrospect is almost definitely not the right way to go, and I won’t even remember most of the things that must have seemed major a day after the last end-term exam now.

Today, we also had the second Metakgp Talk session with two talks. The second talk was by Ashris and Ankita. It was about the intersection between Art and Engineers and how most people are on a spectrum going from Artists to Engineer. It was a GREAT talk and I absolutely loved the oppurtunities for more thinking that it opened. I definitely intend to look at a lot more about the artistic expression that must have gone into the tech products that we all use every day.

The discussion that followed this was almost an informal Metakgp meetup over dinner, and interestingly, it brought up this report about Surveillance in India, for the most part the surveillance reporting that I have seen on the internet has been based on the NSA leaks and mainly US specific intelligence and journalism. That India has such a huge report about these things and that they are implementing a scary surveillance program is disturbing and has essentially made me double down on my own goal of using only products that are end-to-end encrypted and can be trusted, because of the cryptography that powers them.

On a completely different note, There is a project by Himanshu called Muriel which lets you select a folder containing a lot of episodes and subsequently just selecting any one show to play any random episode of that show, ensuring that you are not just repeating the same episodes that you keep watching and also ensuring that the choice is independent every single time. I watch and re-watch and re-re-watch Seinfeld Episodes every single time. I used the following macro to figure out which episode to watch:

echo ‘SEASON ‘$(($(($RANDOM % 9)) + 1))’ EPISODE ‘$(($(($RANDOM % 24)) + 1))

If the episode doesn’t exist, I simply play it again to get another episode that exists! It worked pretty well and it’s almost no extra load at all (Muriel is an electron app packaged in a 45 MB zip of some kind. I don’t know anything about Electron packaging of apps, I believe it is similar to python packaging into Deb files back in the day).

Anyway, I will probably implement a more generic script that will list all the folders, choose one randomly, then go into that folder, select a folder / file there randomly and so on. All Linux and Bash users in general will be able to use this script and it should do a lot of people a lot of good. I know it will do me a lot of good! :smile:

Meanwhile, @SeinfeldTV on Twitter replied to a tweet of mine! EXCITED OUT OF MY MIND (It’s not that big a deal, I know, still excited though):

POST #55 is OVER

Day 54 - 80 hour work weeks; 100 hour work weeks(!!); BURN OUT

2017-04-12T00:00:00+00:00

I found myself thinking about how people survive and put in 80-hour work weeks or 100 hour work weeks. Assuming that you put in 80 hours over the 7 days of a week, that gives you about 11 hours of work each day. On top of this, there is the commute time from your home to your office. That will be atleast 30 minutes for most people anyway. Add 1 more hour for lunch, and that comes to an even 13 hours each day. Say you leave your house at 7:00 AM, you will come back at 8 PM. Sufficiently exhausted to not be able to do anything except eat and sleep so that you can get back up the next morning. And I am assuming that people put in the same amount of time on saturdays and sundays as well, so basically you would never have any holiday! It’s more likely that you will put in MORE time on weekdays and less time on the weekends. Still, WHAT?!

100 hour work weeks are even more insane. To put it all in perspective, a week has 7 days, which is 168 hours. You have just decided to give up 2/3rd of your whole week to work! With 8 hours of sleep everyday, the total number of hours that you are literally not working or sleeping every single day reduces to a meager 3 hours! It is a concept that I couldn’t wrap my brain around. Pop culture has a plethora of references to investment bankers having to put in 80 hour weeks or 100 hour weeks or how Elon Musk did that for years on end. It’s fascinating to me that someone would actually spend so much time working and still enjoy it. Burn out seems unavoidable, and I have read several articles about how it’s going to come and you should embrace it rather than trying to fight against it happening or making it go away.

I have been thinking about all of this because for the past 5-6 days I have been having 16 hour days, I have been awake from 7 AM to literally 1 or 2 AM every single day!

The time that I posted to this blog under #100DaysOfWriting is perhaps the best metric for when I was just about to go to sleep (on most of these days I didn’t go to sleep right after, but instead hung out on the internet for still some more time and then finally slept around 2 am in the morning!)

Tomorrow is the BTP Final evaluation. Wish me luck.

POST #54 is OVER

Day 53 - United Airlines mess, Prep post for the detailed one, Homeland finale (preview)

2017-04-11T00:00:00+00:00

Ah, the United situation is such a total mess. I have been hanging out at Reddit and learning and absorbing so much information the past whole day (my first complete day with Reddit since a hiatus, which I am not sure why I took), I hope to make like a compilation of what to do when asked to leave a plane?

The short answer: Leave the plane.

The long answer will have all sorts of links to comment threads that share a lot of information about what it really means when revenue passengers are asked to leave the plane, why airlines do it, how they approach it, how systematic it is (they have software that will pick out people randomly, and it’s not really truly random in the right sense of that concept, it’s more of a priority based selection algorithm), and finally, WHY is it necessary?. Why should people who have austensibly promised a ticket on that flight be asked to leave a plane?

Of course, this is all in the context of FAA and flying in the US etc, but I am sure that airlines all around the world operate more or less the same way because the airline industry is perhaps one of those industries where standards are borrowed across countries because that makes a lot of sense.

A precursor and probably a prep post for my long post on the issue would contain these:

All in all, it has left me with a much greater understanding of what to do in situations like these.

Most importantly, this:

Once the police are involved, stay the hell out of it. Take videos, make complaints, but do it all after the fact. Same goes for your own interactions with police. When the cop is about to beat your ass, you shouldn’t start screaming that you know your rights. Save that for after the beating ends or else the beating may not end as soon as you would like.

One of those comments was a major hit, my observation that most people who fly only 10-12 times a year try to get the cheapest flight and the only people who wouldn’t fall in the category “people who booked the cheapest flight they could get” would be people travelling for business who are jetting around the world and the country all the time! (My Reddit comment karma just crossed 600!)

Yeah, I think the above list should be enough for me to construct a good post about this. The fact that complex systems like airlines work, has been a fascination for me. They have huge machines going around the world, and each of those machines has millions of small parts any of which could go wrong at any airport anywhere in the world, and they have to deal with the next 400 people who are dealing with their own crises and will have a myriad reasons for wanting to get to their destination in time! It’s incredibly complex, and I am glad they get it right as much as they do.

What United did with this particular incident, was HORRIBLE. It was a barbaric act, they could have done a lot of things differently. I don’t know anything about how these situations are normally handled, but in any case, there’s NO JUSTIFICATION FOR DRAGGING A MAN SCREAMING OUT OF A PLANE. The Internet agrees.

Moving on, more has been said and written about r/place. The latest at sudoscript.com, I haven’t read the article but I have been told that it’s a great article and I intend to read it eventually. I need to be done with my Bachelors Thesis final presentation before that.

2 days to go now. About 33 hours, to be precise. I finally completed the LaTeX part of the report today. A few final touches will make it print ready. Create a Presentation after that, prepare for the presentation, and present Thursday morning! There’s enough time to do all of that, I hope.

Emma is still stalled, right where I left it. I have not been able to read one more page of it with all this time spent on reddit.

And apart from hogging most of my blog space, my time on Reddit also ensured that I have also not been able to talk about Homeland! I saw the Season 6 finale today. IT BLEW MY MIND. I know I have been raving about Homeland a lot lately, but that’s because this season has been like a home-coming of sorts. The episodes have been crispier, each character has more depth, every character pops out of the woods and shows us a side of theirs that no one else knew existed.

I debated writing spoilers in this post, it’s far too early to be honest and I don’t want to write anyway. Eventually, I will write a post about the Finale, one of these days I am going to be struggling for content anyway and then, this will be an easy one to write about!

I am going to call it there, this post has actually run far longer than I intended it to. It’s 1:21 AM now.

POST 53 is OVER

Day 52 - A Hexahedral mesh in Ansys; Reddit Android clients comparison

2017-04-10T00:00:00+00:00

Today was almost like yesterday, continued. More LaTex editing, more Ansys simulations inside Static Structural. It’s been so long since I saw YouTube tutorials for things (I think the last thing I saw them was for figuring out how Python’s IDLE worked! That was 5 years ago, I guess. I had no clue about what a Python shell was.) It’s surprising how easy it is to find Ansys tutorials. It probably tells people that it’s an insanely complex software.

I am sure it’s for good reason, but couldn’t it be made just a little bit easier? All I wanted was a fine hexahedral mesh. I had to watch 3 tutorials to figure it out, and it was actually a very simply fix. I got pulled into MultiZone meshing, and so on. I was able to figure it out, eventually though. And once I did, the stress distribution didn’t change as much as I thought it would. It changed a bit, I guess. I haven’t had the time to even look at the results properly yet.

Which is also the reason I haven’t seen Homeland’s 6.12, Season 6 finale episode yet! I know it’s out, I don’t follow anyone who might give me spoilers about it, I read this one tweet that I am sure I will forget by tomorrow. I have stayed off of Twitter for this reason, I am bound to come up on a Gif made from the episode sooner or later.

That also gives me a good chance to get on Reddit, but I didn’t realise that unlike Twitter which provides a feed that is impossible to get to the bottom of, Reddit’s feed is static until you refresh it. So, it’s like an article that’s constantly changing in the background but you can’t feel it at all. There are no obvious markers anywhere on any subreddit or any reddit android app.

I tried a bunch of those today. I was using the Reddit official Android application for the past 3 days, and it was fine. There wasn’t anything crazy about it, it was definitely not as good as Relay at the speed that it was loading things at or in the intuitiveness of Reddit’s comment threads. Comment threads are some of the weirdest and hardest understand features on Reddit with things like “context”, “show parent” and “mark as read”. Every comment reply gets added as an unread message, it’s very confusing to me.

I tried Boost for Reddit which is another Reddit android client. I was liking it until I realised (about 3 minutes in) that when I am looking at reply 3 to my comment, I can’t see reply 1 or reply 2. The only way to do that is to go to the complete comments thread and then manually scroll down and find my comment somewhere and then dig into the replies.

Of course, Relay for Reddit has the nifty “MINE” feature which lets me switch back and forth between the comments that I posted. This feature is probably the one that I use the most because always, after understanding a story and reading all the material, once I start discussing it’s only my comments and the replies to them and my replies to those replies that matter. Everything else is just background noise that I will catch up on later.

This is a blatant plug for Relay for Reddit. I was around when the Reddit official android app was first released and they changed their API and barred everyone else from using the Reddit icon in their client icons. It was all very aggresive and all, but Relay is clearly still the better one amongst the two!

POST 52 is OVER

Day 51 - Writing tex, music I heard through the day, r/BigLittleLies and bad critics

2017-04-09T00:00:00+00:00

I spent most of today staring at a tex document inside vim. I was listening to some different kind of music throughout the day actually. I started with RHCP’s Stadium Arcadium. Specifically, the three songs Stadium Arcadium, Hey and Snow. I LOVE Hey. “HEY, Oh yeah, How long?”

Then, I switched over to The Neighbourhood’s #icanteven, which is a nice upbeat rap included song. The beat is really good to get you out of sleep and really concentrate on the text on your screen. Or anything else, really. Now, I am listening to Arctic Monkeys. I somehow always end up listening to them. Either AM or Favourite Worst Nightmare or Suck It and See. One of those three albums. Their whole discography has so much variety and the song sequences of the whole album are set in my mind so I know what’s coming before the song plays. It’s a great feeling, knowing what song is going to play next.

While AM and RHCP are pretty popular, I always got the feeling that The Neighbourhood didn’t get as famous as they should have for all their songs except the smash hit Sweater Weather. While I love that song, the top 10 songs on their Spotify page are all good!

Sweater Weather has an incredible 212 Million plays, but it’s really surprising that people didn’t click through the artist and come to this page and play their other songs. One track minds for almost everyone out there, I guess. Anyway, I am glad I went to their artist page and heard their other songs and one of their albums. If you are reading this, and haven’t heard the songs in the above screenshot, go right ahead and listen to them!

Yesterday, I contemplated writing a post about the music I like going in a chronological order, starting somewhere in the 80s but it was too great a project and I would inevitably miss someone or the other out. Even when I was trying to make a list in my head for myself, I was missing out on a lot of artists that I like and listen to very often, like Ed Sheeran, no less!! I will do it some day soon, maybe after Thursday’s Bachelor’s thesis presentation. There should be enough time for that and other such ambitious projects then.

Big Little Lies ended last Monday. I was disappointed about the things they changed from the books, other than that I was glad about the binocular scene at the end and the zippo closing shut, it was probably the woman detective who was doing that. I posted a link to r/BigLittleLies about Season 2 and that link earned me my first Link karma, and about 20 more after that! I didn’t realise that I would miss Perry this much though.

Another link that was posted on this subreddit was this thread about how male reviewers didn’t understand BLL. I read the criticism of the reviews, and then I read the reviews themselves, and I found them to have grossly overlooked the complete show. In fact, the most startling comment in the NY Post review was this:

Instead, we’re subjected to scenes of the S&M sex games played by Celeste and her husband, Perry (a one-dimensional Alexander Skarsgard)

Not even the most distracted viewer of ALL 5 episodes of the show, as the author claims to be, would refer to Perry’s abuse of Celeste as S&M games. To pass comments on a show without even seeing it properly, when it’s your JOB. How do these critics get away with it?

I read both reviews through and noted down all the gross mistakes in those reviews. I won’t call them mistakes because they weren’t mis-understandings of the show, they were presumptions from having watched the show for 5 minutes and then trying to pass judgement on the show. Incredibly short sighted authors and reviewers: Mike Hale and Robert Rorke. I wish they would up their game and write a respectable review that actually HIT the things that were bad with this show, if they could find anything.

Also, I noticed that many articles were asking for a season 2 just so we could have another stellar soundtrack. Season 1 had some of the best BGM in the show, probably after Westworld and Game of Thrones, this is one of the most memorable soundtracks. Although, most of the songs were not originals composed for this show, they fit the situation perfectly in every case and that’s what I care about!

POST #51 is OVER

Day 50 - HALF WAY MARK, tweets_analyzer repository

2017-04-08T00:00:00+00:00

HALF WAY THERE!

I have been writing a daily blog post for the past 50 days. A lot happened and through that time, and I will probably write a summary of all of it once I am totally done with the #100DaysOfWriting challenge.

Today, I ran the x0rz/tweets_analyser repository using a Twitter API key and I must say that the data that it threw out was very very fun!

Of course, I first analyzed my Twitter account: @_icyflame

In the timings that I have tweeted in most over the past 1000 tweets (in 372 days or there abouts), I have NEVER tweeted between 3 am - 4 am and between 5 am and 7 am. I would have believed that I would have probably tweeted at all times of day atleast by now!

The break up for different week days wasn’t interesting. The distribution was almost uniform throughout the week. I had only one Geotagged tweet and that was from Kharagpur! So, I think I am doing pretty well there, security wise.

Most used Hashtags was surprising though. I would have definitely thought it should have been Homeland, but it was GameOfThrones by a narrow single occurence!

Atleast @SHO_Homeland was my most mentioned user. That amounts to some redemption. I do love Homeland much much more than I love the 6th season of GoT.

And finally, the domains of the URLs that I keep tweeting. Most of them are goodreads, considering that all of my activity gets mirrored here this is not surprising. After Goodreads, the common ones are my blog’s domain, Youtube and github.com. None of that is surprising in the least!

I went on to analyse @dhh, @CaseyNeistat, @MKBHD, @eevee. All of these accounts are very active accounts. Their last 1000 tweets came in at the last 72, 128, 85, and 14 days. There’s a lot of insight to be gained about sleep cycles and the people they follow and are closely connected to.

This is a great tool! I have wanted to build something of this sort for git for so long now. I think I already said this in an earlier post here, but I do really want to build this. (In one of Linus Torvalds’ interviews, he said that he ran some analyses and found that most kernel developers now work between standard times like 9-6. I don’t know how he found that out, but it probably took him a few minutes to write the whole thing in C or bash(?). I have a feeling he didn’t use a wrapper like Grub or git-ruby to write that.

I also stumbled onto this discussion on r/sociology about the latest r/place experiment. I think this is the right place to get more insight on what really happened there and what all people could read from it! I saw the Timelapse video. It game me some perspective over the HUGENESS of the canvas and there were so many thing going on at the same time that it was very hard to follow through manually. I wrote down what I thought about some of the major battles that I could notice after watching the timelapse a couple of times.

POST #50 is OVER

Day 49 - Western Groups in the wee hours, MFQP PR 30, Homeland finale

2017-04-07T00:00:00+00:00

Today is going to be an interesting day. My day isn’t over yet, not in theory anyway. The Western Groups Interhall event is going to happen in about 4 hours(?) from now. There’s no way to know when it would happen.

I finally opened PR #30: Add ability to pre-fill the search bar from URL param. This particular change has been on my fork of this repository for quite a long time. It should find it’s way into the repository’s master branch soon. The most important functionality that it would add to MFQP would be the addition of mfqp as a search engine in Chrome! So, typing mfqp TAB query ENTER should complete the search. Earlier, the process was to go to https:\/\/qp.metakgp.org and then type the query into the search box manually. This would make using MFQP slightly more pleasant!

The Homeland season finale is going to drop on Sunday! I am so excited about that, especially because of all the new plot points that were introduced in 6.11. Rupert Friend has been doing the rounds on all the morning shows and the late night shows promoting and answering questions about Homeland and this is the first time that he has started levitating towards the lead actor Emmy and not the supporting actor Emmy which he was nominated for once.

The other names being thrown up are Sterling Brown’s Chris Darden from People vs. O J Simpson and Reese Witherspoon’s Madeline from beloved BigLittleLies! All three played complex characters and unfortunately, I am too biased towards Claire Danes to support anyone except Rupert Friend, but I think they might all have a good chance! (And there will obviously be people from TV series that I haven’t watched, right? RIGHT?) The Emmys are in September this year which is an incredibly long way off, so everything right now is too early anyway.

Unfortunately, I haven’t made any progress with Emma ever since. I have started working on my final Bachelors thesis report now. It’s a latex document that’s already been prepared, template / fonts / external packages wise. I just need to write stuff and then run the build.sh script. I have a similar setup for all my Latex projects. This script takes care of annoying stuff like the deletion of the log and aux files that are created when a TEX file is built with xelatex.

I removed gnome-keyring yesterday, things are still fine. The password entry from inside the terminal isn’t too annoying at all! And I use GPG so rarely that it doesn’t matter anymore.

POST #49 is OVER

Day 48 - Updating my RSA keys to 4096 bit after reading Gert's article

2017-04-06T00:00:00+00:00

I came across this article yesterday, which urges everyone to move from RSA to Ed25519, EC-based public keys. Totally legit article, and covers all possible corner cases.

I was using 2048 bits RSA keys till now, I have updated them to a 4096 bit key now. An Ubuntu 14.04 LTS server refuses to accept Ed25519 keys. When specifying the pubkey protocols that it understands, it says rsa, 535, blen and one other suite. The OpenSSH version is 6.6, and the post says that anything about 6.5 should support Ed25519 keys. I need to investigate this.

There is a sherr size difference in the RSA keys and the Ed25519 keys. Although both afford the same amount of security, the Edwards Curve keys are much much smaller in size.

A 4096 RSA keypair is 4184 B long. The private key is by far the larger file. Now, come to the Edwards CUrve and a whopping 10-fold decrease. The keypair is 570 B long, yes, that’s both the public key and the private key!

Although support for Ed25519 is patchy right now, I am sure they will take care of this in the future versions of OpenSSH.

One slight issue that I ran into was when I removed the old RSA key from the server and tried logging in with the brand new 4096 bit RSA key, it said “sign_and_send_pubkey: signing failed: agent refused operation”. This was because the SSH Agent (which was GNOME Keyring in my case, a default Ubuntu installation) was refusing to sign this 4096 bit RSA key. A quick sudo apt-get remove --purge gnome-keyring and system restart removed that completely, and now the agent is nothing fancy and asks for password from within the terminal itself.

I am obviously holding on to my old key just yet, it will probably take a long time before I can delete those files having completely switched to 4096 bit keys. Probably, not even then, there just might be an esoteric old and long forgotten computer with Ubuntu installed on it, where the legacy keys are added. There’s no harm in keeping the legacy keys as long as you are not going to (even accidentally) add them to a production server.

POST #48 is OVER

Day 47 - Updated About page after 2 years, Homeland 6.11 complicates the TV show more

2017-04-05T00:00:00+00:00

I updated my About page. I should have done that a long time ago, I last updated that page on March 31, 2015. That’s two years and 1 week before today! A HUGE amount of time in which my outlook about a lot of things has changed. I glanced at the diff again and I noticed the change in the tone of the text in that page. Having written history to compare yourself to past selves is an interesting experience, sometimes my past naivete intrigues me in the same way that understanding after learning something, it’s hard to imagine a time when didn’t know that thing. I am pretty sure that whole situation is called something, I don’t know what the search term should be though, so I am going to let it go.

AH! I saw Homeland’s 6.11 episode. It created some new troubles for Carrie and Quinn and ended again with a bang. Apparently, Claire Danes said over the past few days when she’s been doing the rounds talking about the season finale that the finale will leave the viewers happy. At this stage, I have absolutely no clue how they can make anything happy, almost everyone is in a bad place except for Saul and Keane who are slowly understanding the situation that they have landed themselves in. Max is stuck, Carrie and Quinn are too, O’Keefe has some weird pull over audiences and cleverly edits sound and video to make it sound like he is being villified. It’s a HUGE MESS, and they need another season JUST to explain the stuff that’s been going on.

I think this season was mostly about Quinn and most importantly, Quinnn’s past. His staging house for Mid-East operations, Robert Knepper who appeared for only two scenes in the whole show and has been missing ever since, the whole Dar Adal play and what that end-game is, there are so many moving parts! The stage for season 7 is beautifully set, unless they decide to shut everything down in the finale. That’s probably not going to happen!

The Big Little Lies finale dropped as well. A one-hour episode to cover all the ground that was left uncovered till episode 6, this could have easily been a 10 episode mini series. Obviously, I just want to increase the number of episodes because I really liked the book and would like to see an unhasty conclusion to the show with Perry and all! (Alexander Skarsgard does such a great job playing that character, he’s 7 feet tall and his scenes with the twins are the BEST!)

I haven’t seen the episode yet, I hope they did justice to the explosive finale that I am looking forward to. Scenes at the end of 1.6 showed that the setup for the climax is the same as that in the books.

POST #47 is OVER

Day 46 - Re-watching Seinfeld on Amazon Prime - The Stall and The Stake-Out

2017-04-04T00:00:00+00:00

I set a goal to read 40 books this year on Goodreads at the start of this year. Last year, I read 30 and I did that pretty peacefully (December was totally free and I crammed about 5 more to go over the goal to 35). Now, I am “on track”, which means that if I don’t finish the book this week (week 14 of the year) then I will be “behind the schedule”. THAT is going to happen. This sounds so much like the iconic Taken scene:

This was a completely new take to action cinema, the hero always used to RUN for the person being kidnapped, reach there on time, or after. Then, fight everyone there to escape, this was Liam Neeson accepting that she would be taken and that he would then go and find her! I was taken the first time I saw that scene. ( WINK, WINK )

I watched the Amazon Prime Comedy specials of Naveen Richards and Varun Thakur, I am in the trial period though, so this is going to end. A gem that I did find while browsing Prime Video was Seinfeld with HINDI subtitles! When I turned them on and followed them along, I realised that they were really accurate! I wonder who had that job at Amazon: write Hindi subtitles for the greatest comedy show ever! And Kramer’s dialogues in Hindi? The next thing I am going to do after writing and posting this post will be to check out the episode “The Stall” with Hindi subtitles.

Particularly, The Stall has an exchange between Kramer and Seinfeld that starts with Kramer saying “AM I? OR AM I SO …” That scene got overshadowed by the other main premise of that episode which is the thing between Elaine and Jerry’s girlfriend and the voice of Erica / Jerry’s girlfriend being tattooed on both Elaine and Kramer’s brain for radically different reasons. Anyway, it’s a great episode and worth watching twice or a 100 times if time would allow it.

Another thing that I noticed while watching Episode 1.2 The Stake Out was the scene in which Elaine’s friend at Pamela’s birthday party, whose law firm George and Jerry stake-out. When that friend leaves, all the men at the table follow her along until she reaches the door! On my first time watching this scene, I was only looking at Jerry and the women going out the door and her boyfriend, the other men were totally out of focus to me. THIS is the pleasure of rewatching a great show!

POST #46 is OVER

Day 45 - r/place shuts down after 72 hours, WHY was it created?, Emma is an ENIGMA

2017-04-03T00:00:00+00:00

If you didn’t know about r/place, I will give you a lowdown on it. It first started as an empty canvas apparently. When I went to it, there was a lot of pixel art there already. Anyone can place a pixel anywhere, with one catch: you can only place a pixel once every 10 (later 5) minutes. This is how the canvas looked at the very end, about a couple hours ago when r/place crashed and the whole project was ended. Initially, I thought this was a social experiment to prove that there are more constructive art-makers on reddit than there are trolls who only want to destroy everything good and honest. Was it, though? I couldn’t find the first post on that subreddit, which I am somehow convinced would have consisted of some rational of building this thing. I will keep trying to find out why this thing stayed alive for 72 hours and then just stopped.

It would be safe to say that people did indeed come together and made something great starting from an empty canvas spewed with NSFW messages to the final form of the canvas which had a lot of flags, the GNU/Linux penguin, Counter Strike logo, Tesla logo and SO much other art that wasn’t created by one person. This was a fun thing that was around for three days and I got to experience it in real time! And not in retrospect. There have been so many 2-3 day trends that I have missed out on simply because I was tuned out and doing something else and was late to the game.

Inched a little bit further in Emma, finally she starts to “get” Elton! Harriet is soooo being taken for a ride! But there is this particularly fascinating and disturbing paragraph. Emma is daydreaming about Frank Churchill, who is a mysterious character who’s never been seen by anyone in Hartfield, and herself being seen as a couple in everyone’s eyes and RIGHT AFTER that, Weston (Miss Taylor’s husband) calls her up and tells her that he thinks Frank and Harriet would make a perfect match and she doesn’t bat an eyelid before answering! Not even in her inner monologue! HOW?! This is disturbing because this is weird behaviour, if you are daydreaming about someone and their parents want you to arrange a match for them with a friend of yours, what would your reaction be? (This is totally only going to happen in a late 18th century book where you call your father Sir and all the conversation is painfully formal and to the point and asking after other people obediently because that’s what you do.)

Emma Woodhouse is an ENIGMA. Frank Churchill is also an ENIGMA.

POST #45 is OVER

Day 44 - Learning vim, installing plugins and spending time on tooling

2017-04-02T00:00:00+00:00

So, yesterday I wrote a couple of scripts. These are mainly to make the counting of the posts easier. Till now, in the month of march the post number was simply the date + 11, but now and going forward it will be pretty hard to keep track of this number and at the same time save a few keystrokes in opening the editor and typing the number in, removing the TODO tags from the template. Anyway, the ruby script was fun to implement. It probably doesn’t need to be this big(?), I just wanted to be sure that I would get the right date for the post on the first attempt itself and I won’t have to edit the name or something like that. You should keep working on tooling, until you feel good about everything that you have (which will never really happen)

This is exactly the approach I took to vim, I used to use Sublime Text full time until I decided to switch to vim. For the longest time (about 3 months), I used vim with absolutely vanilla settings, I spent no time at all on setting up my preferred keyboard shortcuts or installing plugins even though the vim plugin ecosystem was something everyone raved about.

But eventually, I found vimawesome, which is a very pretty catalogue of all the plugins that are available for vim. I installed Vundle, then I installed each of the plugins that I now have. I think nerdcommenter is a plugin that I really needed but couldn’t figure out how to work for about a month, but then I scoured the readme, tried different key combinations and then I finally understood the readme. The fault was all mine really, I had no clue what the <leader> meant back then.

There were some other plugins that came and went away quickly, NerdFileTree was one of them, although I liked that, it couldn’t even compete with the convenience and the sheer power of CtrlP. CtrlP is my second favourite vim plugin, and it’s also there everywhere else, any IDE, any text editor. A lot of people surprisingly don’t know about this particular functionality and spend quite a bit of time going around in their terminal or even more time navigating around Nautilus. CtrlP exists: USE IT, you will LOVE IT

Right after those two plugins, I am going to put vim-system-copy which I installed very very recently and vim-multiple-cursors which again is a plugin that’s been installed for quite some time, but I learned to use properly only a few months ago. Both these plugins have made life even more easy for me! (Especially inside tmux where the tmux-yank plugin has to be used to copy text. Yeah, unnecessary complexity, I am sure there’s a good reason for that to be the case.)

Syntax highlighting, vertical and horizontal splitting of the screen into panes and all that common editor stuff is cool. TL; DR Spend a weekend getting vim to do what you want an editor to do for you, spend another weekend’s worth of time doing full time work with vim even if it is slow and painful as hell and requires a lot of searches in the browser, DO IT. Eventually, it will all be muscle memory and you will have picked up the amazing skill of being at ease no matter what computer is thrown at you.

I actually wrote a lot of the above paragraphs in a cover letter that I submitted for an internship position at Basecamp, for Summer 2017. I didn’t get in, I did get a phone interview with Noah Lorang (who happens to be the Data team head. WHAT?!) and I wonder if the person who was screening the cover letters was influenced by the couple sentences I wrote about how vim made me all powerful when I sat down at a new computer or ssh-ed into a droplet somewhere.

I have also had the pleasure / discomfort of using vanilla vim forcefully, I always keep a basic vim configuration handy at this gist, who’s short URL is stored on my phone as a contact. You are probably thinking that this is too convoluted, but it’s a really convenient setup. (wget short-url when on a new computer)

POST #44 is OVER

Day 43 - MISSED; Clock issues on Windows-Ubuntu Dual Boot

2017-04-01T00:00:00+00:00

Ah, this is going to be my sixth MISSED post. The irony of it all is that yesterday, I really had a topic to write about. A topic that a friend asked me about earlier in the day and one that I have faced many times myself. Uh, irony.

Managing your Clock on Windows-Ubuntu Dual Boot

Switching over to Windows and seeing the wrong time or switching to Ubuntu and seeing the wrong time, it’s all pretty irritating. It keeps happening, there’s no easy way to fix that, this is my recipe. It’s pretty easy, I think.

Turn off the Update time from Internet option on all your operating systems
Boot into Windows, and open the registry editor. (WinKey + R followed by regedit in the text box)
Edit this key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\TimeZoneInformation
Right-click in the right-side panel and select New > DWORD Value. Create the key named RealTimeIsUniversal and give it a value of 1.
Now, restart your computer and boot into Ubuntu. We will set the Hardware clock to UTC time.
Open a terminal, you need root privileges on your machine
Let’s start with the basics. Run sudo hwclock --show, this should show you the current time that’s stored in your hardware clock
Set this to the current UTC which should be available to you through your phone or the Internet. (Read further to see how to do this from a great GUI)
1. hwclock supports the GREAT feature of copying over time from the system
2. So, open the calendar / date-time GUI application and set the proper time there.
3. Make sure automatic updation of the time from the internet is turned off
4. Once you have set the proper time for your timezone in the window, you can run sudo hwclock --systohc. This will copy over the current system time to the hardware clock.
5. You should also probably run sudo hwclock --utc to make sure that the hardware clock is kept in UTC
Open the file /etc/default/rcS and UTC=yes to that file
You are SET now!

This guide serves as an enhancement to the two pages that I have in my bookmarks bar under the folder Clock issues (Yeah, right, that’s how big a problem this was)

Windows-Linux Dual Boot System Issues on mikebeach.org

Examples of using hwclock on thegeekstuff.com

That’s the end of my tutorial for the most part, a couple of quick points about this setup.

The main problem here is that Linux interprets the Hardware clock to be set in UTC whereas Windows assumes it to be in Local time by default, this creates the initial inconsistency.

Automatica time updation from the internet probably changes the hardware clock right after startup and right before shutdown instead of changing it once? I don’t know anything about this, but I have a feeling that this also works differently in both operating systems. Simply turning it off and adjusting for the drift of the hardware clock every year or something as infrequent as that will ensure that the setup is simple! (This also enables freely changing the timezone on whichever operating system, that should not affect the hardware clock at all)

POST #43 is OVER

Day 42 - Taping your webcam, open ports, the WHY of it all

2017-03-31T00:00:00+00:00

I have thought long and hard about the topic that I plan to write on today, Privacy. I taped my laptop’s webcam about 2 weeks ago, it afforded a renewed notion of security, for the past 2 years, I have had it obscured because the lid of my laptop is mostly closed. But now, I am all in. When I think about the doors that I have left open for attackers, I want to be sure that there’s nothing obviously wrong that I am doing. Before my recent Ubuntu re-install, I had the package openssh-server installed on my primary computer! YEAH. Port 22 was open, it was behind a router and I used it to connect to my computer through my phone.

Let me say that again: I had an RSA private key on my phone that could be used to get into my computer. Obviously, I didn’t know how insanely dangerous and stupid of my it was to have this kind of a setup, but I used to have that, and now I don’t. That’s partly because of all the articles that I read online about privacy and really important people’s email getting leaked or copied or hacked into or stolen, and partly because I finally understand how easy it is for someone to mount an attack, all it takes is a bit of social engineering (talk to the victim, get some details out of them, have access to their phone in an unlocked state for a short period of time) and the determination to want to get in.

Specifically, I think I should mention Robert Heaton’s post about cloning Facebook sessions using the Chrome extension EditThisCookie, which I tried myself with an incognito browser and my actual session and IT WORKED! A running ratchet that would change the session key for encryption every few minutes sounds like something that Facebook should have implemented? It all sounds really shady to me, because there were literally two steps to the whole process: 1. Install the extension 2. Copy the cookies and send them to yourself. The whole process would take about 5-9 minutes on a decent Internet connection with a good mouse and keyboard, assuming a logged in email account which is totally legit.

Obviously, the Snowden leaks, NSA apparently being able to switch on webcams without turning on the indicator lights on many versions and the Black Mirror S03E03 were all powerful contributing factors. (Mr. Robot didn’t have any privacy specific implications yet, but the next season might? should? We will see!)

P.S. See that black mirror episode if you haven’t already! The show was taken over by Netflix for the third season, and the production has changed and the difference between British production and Netflix production is rather visible! Nonetheless, all seasons of that show are GREAT.

POST #42 is OVER

Day 41 - MISSED; Progress with Emma; Comparison with Aisha;

2017-03-30T00:00:00+00:00

Ah, finally, I made some real progress with Emma (Austen). The characters are realy panning out, this book is far easier to read than I would have thought of, she writes in a simple style, and there are a few complex sentences here and there, but most of them are conversational and easy to understand!

I learnt some archaic spellings that I believed were typos, hilariously. choose was spelled chuse and show was spelled shew. Very very surprising!

I am almost immediately reminded of this movie:

A 2010 Hindi movie with Sonam Kapoor as Aisha and Abhay Deol and some other good actors. Aisha is definitely inspired from Emma, she’s rich, good looking, doesn’t want to get married and has a father who indulges every one of her insane plans about the new girl that she befriends and is determined to match with a friend of hers from her own social circle, while that girl is in love with some other boy from her life before Aisha. It’s an OKAY movie, one of the songs is really good, and it’s an over-all not too heavy, chilled out account of a rich, young girl’s life. I wonder if this novel was credited in that movie. Although this kind of a story is quite common and it doesn’t need an Austen to tell us about it, the specific circumstances are too similar.

The ages of most characters have been revealed now. Knightley is 37-38, Elton is 27 and Harriet is not even 18 yet. Emma’s age hasn’t come up just yet, but I guess she would be in her twenties because she is definitely older than Harriet and Elton’s admiration (love?) for her suggests that she might be his contemporary. Knightley is the voice of reason in the book, that’s Abhay Deol character’s in the movie. At the end movie-Aisha and movie-Abhay Deol get toether and get married or something, I wonder if that’s even possible with this book. Just have to wait and see what is going to happen.

I also watched the finale of People v. O J Simpson. I didn’t know the outcome, I don’t want to spill any spoilers for the people who haven’t seen the live news coverage or the show yet. I can’t say anything about it because like one masterful scene in the show where detective Mark Fuhrman is asked:

Cochran: Did you falsify any evidence in this case? Fuhrman: I assert my Fifth Amendment privilege

This was a masterful stroke from Shapiro and Cochran.

In some more TV news, an Emmy for Rupert Friend is in the air? He had one nomination in 2013, probably for the first season as Quinn where he was THE BADASS TO BE. Now, he has a much deeper role and is pulling it off INCREDIBLY well! I wish he gets an Emmy! #EmmyForRupert

POST #41 is OVER

Day 40 - Tweets analyser, Git analyser idea

2017-03-29T00:00:00+00:00

I started reading Emma (Austen) today! The comforts of a story being linearly narrated are manifold, I missed that! (Both my previous books were confused narrations which switched here and there and totally left me in reading-epilepsy, I liked that too, it’s good to be back nonetheless) This story is also written in late-18th century English with the whole flourish and the Keira Knightley and Rosamund Pike-esque characters. Emma Woodhouse’ introduction was solid, Mr. Knightley just entered the scene!

(That applies to Tuesdays this semester)

Meanwhile, fun fact, most of my previous posts have been really late, about 30 minutes before midnight or hours after midnight. I love Git for this extent of statistics reporting that’s built into it. For the longest time I have wanted to build some kind of analyser or a gem of some sort which will use a ruby wrapper for the Git binary and show some kind of “cool” statistics about projects. Specifically, I wanted to use it to find out how “active” an open source project was: how many new people contribute each week, how many contributors return and contribute regularly, what kind of times do they work in between. I think the last metric is especially important in finding out if a project is something that the contributors do for work or pursue as personal projects.

This could also be done for Github users on their public repositories to see what type of code they are writing and what time they are writing it in. All of this is great talk, but

Talk is cheap

So, this still remains in the “ideation” phase and never made it into the real world. I found this repository through this article: x0rz/tweets_analyzer. Given a username, it tries to find some stuff about the user, specifically it uses the timezone, the times that the user often tweets in, the users that are most interacted with and retweeted: this probably shows people that the user has turned on notifications for (?) because it’s next to impossible to continuously retweet someone unless you are pinged each time they tweet something. Twitter is an incredibly hard platform to just stay focused on one thing, if you follow a few 100 people and then you scroll down for about 2 minutes, your feed updates with 4-5 tweets and the blue dot next to the “HOME” icon is hard to ignore and just keep browsing. This is how FOMO works, yes? (Note to myself: I should add FOMO to the awesome-social-science list that I maintain)

POST #40 is OVER

Day 39 - 8 BILLION passengers are carried every year by the Indian Railway. WHOA!

2017-03-28T00:00:00+00:00

Okay, I can feel this being a short post. It’s just that I am in the phase between two books where I haven’t committed to anything just yet. Emma (Austen) is waiting to be picked up. That book is in the public domain, and the people at Amazon are really nice to put it up on the Kindle store and save everyone the trouble of having to download it and copy it over to the Kindle!

An interesting thing that I discovered today was the total number of passengers that use the Indian Railway system each year. The number is HUGE. It’s HUGE, it’s like nothing I would have imagined. Before seeing the figures, I made a rough estimate of maybe half the population of India which is about 600 million.

The actual number was 8 BILLION passengers, per annum. Yeah, that’s 500 million more than the TOTAL POPULATION OF THE EARTH. It’s a NUMBER that I can barely even make sense of, 8 times the population of India! (That number is the number of passengers carried in rail transport in 1 year)

Surprisingly, two things come up here:

Japan has 9 BILLION PASSENGERS carried each year. HUH? The population of Japan is about 120 million. That’s 10 times less than India’s population. Also, the total area of India is about 9-10 times more than Japan’s area. How can the passengers carried be more? It’s extremely intriguing.
China has ONLY 1 Billion passengers each year. That’s barely more than their own population, do the Chinese travel less? Or do they travel more by flights and less by trains? In either case, this is surprising. I expected this trend to directly reflect the population, I mean in absolute terms the population would be expected to be the defining metric here!

But despite this, China happens to have the most passenger-kilometers product at 1196 billion. India follows at 1147 billion and then a STEEP drop to 260 billion in Japan. So, every year Japan, more people are travelling but they are travelling for shorter distances. Whereas in India a similar number of people are travelling each year, but travelling GREAT distances. Whereas in China, very few people are travelling by trains but they are travelling a GREAT amount of distance. Naively, dividing these two numbers gives us an average of 1000 km travelled by each passenger, assuming that every person in China travels a similar amount of distance each year.

That number happens to be pretty close to the 770 passenger-km per capita that India has. So, for every person in India, an Indian Railway passenger can travel 770 km. That metric doesn’t make sense to me, but it could be an interpretation issue. (I am thinking of it as “man-hour” which might not be the right way to think about things like this)

POST #39 is OVER

Day 38 - Completed Handmaid's Tale - A dystopia with great writing

2017-03-27T00:00:00+00:00

Finally, I got the 1 hour I needed to finish Handmaid’s Tale. A detailed review is up on Goodreads: here.

After finishing this, I got to thinking about whether I should compare this with the other dystopias that I have read. The most influential of these was 1984 (Orwell). A surveillance state with some horrible restrictions on everything and horrible food for everyone who was not a proper Inner Government official. Gilead is a different world, of course. There are stricter observations, but almost no surveillance. Instead this is replaced by a complex heirarchial system among women that ends up in all of them keeping a wary eye on each other for possible signs of betrayal, laziness, non-adherance to the routine preset by the Wife of the household, etc. There are many quote worthy lines in this book, mostly platitudes but some really profound lines about life and controlling people and keeping people in check by not letting them think, etc.

Another thing that I didn’t talk about on the review on GR was that the writing in this book is good! Much much better than you would expect from a plot-oriented dystopia. The author has gone out of her way to include lines like these:

Yet it isn’t waiting, exactly. It’s more like a form of suspension. Without suspense. At last there is no time.

Or ones like:

Fatigue is here, in my body, in my legs and eyes. That is what gets you in the end. Faith is only a word, embroidered.

Or this last quote from the early few chapters in the book.

Tell rather than write, because I ahve nothing to write with and writing is in any case forbidden. But if it’s a story, even in my head, I must be telling it to someone. You don’t tell a story only to yourself. There’s always someone else.

All in all, this book is a dystopia with some great writing! It’s the first book that I have read that fits that particular description. I will definitely not be reading another dystopia right on the back of this one because of some unpleasant experiences in the past when I read two Orwell dystopian novels almost back to back and had to stay away from Dystopia for almost a year before I could start reading Brave New World (Huxley) which is not even a proper dystopia, but it’s an undesirable world in any case.

Next up: Emma by Jane Austen. I was absolutely not joking when I said that I would like to know why Amy Dunne thought that reading Austen makes for better men that would be compatible with her. I aim to find out! Emma is a 470 page book! It’s one of the longer ones and it is definitely not a wise move to pick this up right now when there are so many things coming up (Western Groups, Bachelors Thesis Project Presentation, End Term Exams etc) but I am going to try to start reading anyway.

I say “try to start reading” because I tried to read Sense and Sensibility and I couldn’t. Reviews on SnS (Austen) said that Emma was the the book to read if you are just getting started with Austen! That sounded right to me, and I have the habit of trusting online reviewers. No apparent reason. They took time to write a reply to my comment or to write the review that they did on GR, there’s no conceivable reason for them to lie about it unless they are just some insginificant trolls or people who spam sites like GR for fun.

POST #38 is OVER

Day 37 - A brief history of my blogging past and the tools I used

2017-03-26T00:00:00+00:00

There were some serious production issues in the past couple of posts. The images were not properly downloaded (I downloaded them using wget and that didn’t work out), titles were not right, categories weren’t added. I have fixed all of that now, there was even a spelling mistake, a hilarious one too. I had mis-spelled synced as sinked, I didn’t even notice it when I was typing!

Anyway, the quality of everything should improve from here on out, I am going to visit the post and skim it right after posting. I think I should talk about why I am using Jekyll for this whole series and also as my primary blog when there are much simpler alternatives like Medium out there.

I have been trying to blog for a while now. I first had a Wordpress blog on Wordpress.com. (I logged into it after 3 years with just one password attempt. MEMORY GOING STRONG.)

Anyway, I have just been seeing my posts, and there is a lot of text on that blog. I used to write quite a bit about TV series and things that happen inside IIT Kharagpur. This was about 4 years ago when I had a slower laptop. It was a Toshiba and it choked on running Google Chrome and EiskaltDC++ together. That laptop was the reason that I was using the terminal for almost everything from launching videos in VLC to editing with vim instead of Sublime Text because it was too slow. (It had a Pentium processor, I think with 2 GB RAM. OLD!) Back then, jekyll was just becoming popular with the whole Github pages thing being rather new. So, I found a good theme and got started with publishing on GitHub. There was maybe a 2 year hiatus after which I started this particular series and started posting regularly. In the time in between, I posted semi-regularly, about once in every 3 or 4 months. Wordpress archives [I can’t seem to get a list of all my posts on wordpress, only monthly archives]

In between this, I also used Tumblr. I really wanted to like it. It looked good on browsers and on mobile, I could repost stuff from blogs I follow and post them there. This blog was mostly quotes from the ASOIAF book series which I was reading for most of 2015 and the early part of 2016. I also posted some normal write-ups about the status of my life. I never used the mobile app for posting anything, only for browsing other blogs that I followed, but the interest faded away soon and I stopped posting. I wrote about 40 posts over a period of 10 months from November 2015 to August 2016. The most number of posts were in the first month, and then it came down exponentially. Archive of Tumblr

And, now I am back here since August 2016, I had not written anything before I started this series. I felt really bad about not writing. In March 2016, right before starting my internship, I promised on Medium that I am going to start writing more.

So, I read this essay on medium, and realized that it applied to me. Not the not reading 100 books a year part, I absolutely love reading and will continue to read a lot, but I should start writing, despite not being good at it and all that, one thing that made absolute sense was that I should start writing.

I am sure it was just laziness that caused me to not go through with this particular plan of mine. But I would say that I still had the last laugh, even if it was 9 months later than I had said I would start, I started and now I am 37 days in! GREAT!

TL; DR I used Wordpress, then I used Jekyll for some time before switching to Tumblr for 9 months and then not writing anything for 4 months and finally starting this series on Jekyll.

A brief history of my half-assed past attempts at blogging

Duration	Where?
November 2013 - September 2014	Wordpress blog
September 2014 - October 2015	Jekyll (this blog)
November 2015 - August 2016	Tumblr
August 2016 - February 2017	The Hiatus
18th February 2017	DAY 1

I will now push this post and check if the table was rendered properly. I am 60% confident that it won’t render properly. It has never worked for me on the first attempt! It’s always been a struggle.

EDIT: The 40% won out! I was finally able to write a Markdown table and have it rendered the way I wanted in the first attempt!

POST #37 is OVER

Day 36 - Finding a solution to Ubuntu 16.04 LTS' "mouse not working" problem

2017-03-25T00:00:00+00:00

I am reading Force and Opinion, an essay that was posted on Metakgp Slack, when I posted my surprise at having found the Internet Shutdowns in India. I am still reading it, but it sounds like a solid thing to talk about in another post. I should maintain a TODO list of topics that I want to talk about. (I am now switching to Trello to create that list)

The mouse thing happened again. This Ubuntu was started only about 3 hours ago. This is extremely frustrating, I don’t want to go through the pain of re-installing Ubuntu all over again with the right configuration and I don’t want to have to keep restarting the PC. That would defeat the whole purpose of having a GOOD COMPUTER! I need to find a solution to this problem, RIGHT NOW.

AND FOUND IT! Restarting Laptop’s USB PORTS WITHOUT restarting computers Yet another lesson in effective searching of the internet! I went through the dmesg -w output right before the mouse stopped functioning and it didn’t tell me anything more than I already knew. Does it ring a bell?

[   72.058426] [drm:intel_pipe_update_end [i915]] *ERROR* Atomic update failure
on pipe A (start=4851 end=4852) time 545 us, min 763, max 767, scanline start
756, end 783
[  121.341228] r8169 0000:01:00.0 enp1s0: link up
[  121.341237] IPv6: ADDRCONF(NETDEV_CHANGE): enp1s0: link becomes ready
[  197.470587] IPv6: ADDRCONF(NETDEV_UP): wlp2s0: link is not ready
[ 4505.275283] perf: interrupt took too long (2540 > 2500), lowering
kernel.perf_event_max_sample_rate to 78500
[ 5778.702948] perf: interrupt took too long (3259 > 3175), lowering
kernel.perf_event_max_sample_rate to 61250
[ 6937.288611] usb 1-1.3.4: USB disconnect, device number 6
[ 6965.598679] usb 1-1.3.4: new low-speed USB device number 8 using ehci-pci
[ 6965.710831] usb 1-1.3.4: New USB device found, idVendor=093a, idProduct=2510
[ 6965.710835] usb 1-1.3.4: New USB device strings: Mfr=1, Product=2,
SerialNumber=0
[ 6965.710837] usb 1-1.3.4: Product: USB Optical Mouse
[ 6965.710839] usb 1-1.3.4: Manufacturer: PixArt
[ 6965.714332] input: PixArt USB Optical Mouse as
/devices/pci0000:00/0000:00:1d.0/usb1/1-1/1-1.3/1-1.3.4/1-1.3.4:1.0/0003:093A:2510.0005/input/input18
[ 6965.715236] hid-generic 0003:093A:2510.0005: input,hidraw2: USB HID v1.11
Mouse [PixArt USB Optical Mouse] on usb-0000:00:1d.0-1.3.4/input0
[ 7765.071883] perf: interrupt took too long (4134 > 4073), lowering
kernel.perf_event_max_sample_rate to 48250
[ 8994.057705] usb 2-1: reset low-speed USB device number 2 using xhci_hcd
/[ 9175.482422] usb 1-1.3.4: USB disconnect, device number 8
[ 9184.631763] usb 1-1.3.1: new low-speed USB device number 9 using ehci-pci
[ 9184.748086] usb 1-1.3.1: New USB device found, idVendor=093a, idProduct=2510
[ 9184.748089] usb 1-1.3.1: New USB device strings: Mfr=1, Product=2,
SerialNumber=0
[ 9184.748091] usb 1-1.3.1: Product: USB Optical Mouse
[ 9184.748092] usb 1-1.3.1: Manufacturer: PixArt
[ 9184.750943] input: PixArt USB Optical Mouse as
/devices/pci0000:00/0000:00:1d.0/usb1/1-1/1-1.3/1-1.3.1/1-1.3.1:1.0/0003:093A:2510.0006/input/input19
[ 9184.751181] hid-generic 0003:093A:2510.0006: input,hidraw2: USB HID v1.11
Mouse [PixArt USB Optical Mouse] on usb-0000:00:1d.0-1.3.1/input0
[ 9482.951029] usbcore: deregistering interface driver usbhid
[ 9515.715675] input: Barcode Reader  as
/devices/pci0000:00/0000:00:14.0/usb2/2-1/2-1:1.0/0003:13BA:0018.0007/input/input20
[ 9515.775777] hid-generic 0003:13BA:0018.0007: input,hidraw0: USB HID v1.10
Keyboard [Barcode Reader ] on usb-0000:00:14.0-1/input0
[ 9515.779015] input: Barcode Reader  as
/devices/pci0000:00/0000:00:14.0/usb2/2-1/2-1:1.1/0003:13BA:0018.0008/input/input21
[ 9515.835781] hid-generic 0003:13BA:0018.0008: input,hidraw1: USB HID v1.10
Mouse [Barcode Reader ] on usb-0000:00:14.0-1/input1
[ 9515.838679] input: PixArt USB Optical Mouse as
/devices/pci0000:00/0000:00:1d.0/usb1/1-1/1-1.3/1-1.3.1/1-1.3.1:1.0/0003:093A:2510.0009/input/input22
[ 9515.839827] hid-generic 0003:093A:2510.0009: input,hidraw2: USB HID v1.11
Mouse [PixArt USB Optical Mouse] on usb-0000:00:1d.0-1.3.1/input0
[ 9515.839857] usbcore: registered new interface driver usbhid
[ 9515.839858] usbhid: USB HID core driver

The point 6937.288 is where the USB stopped functioning. See how the existing mouse (connected as device 6) was disconnected and then reconnected as device number 8. This lead me to think that if only I could restart the USB module in the kernel, it would detach and reattach ALL the USB devices. That would be the best! I have my keyboard and the mouse connected on USB ports. A quick look at the AskUbuntu answer and the modprobe man page increased my confidence in the command posted there! So, this was the solution I was looking for! This will go right next to my existing restart network manager alias in my dotfiles. GREAT, this solves a huge inconvenience. This happened yesterday while I was watching a movie and wanted to download the subtitles for that movie. It was an incredibly high and unexpected amount of pain. I didn’t anticipate that.

Okay, that has itself become a long post (probably because of the dmesg output that I posted but it was totally necessary!) While I was writing that sentence, it happened again (weirdly, the frequency seems to increase after fixing it once. But frankly, I don’t care now, I will just continue to run modprobe while researching the problems that this could have. I can’t think of any really, adding and removing modules from the kernel is NOT that catastrophic really.

Okay, I am really calling it here now.

POST #36 is OVER

Day 35 - MISSED; Internet Shutdowns in India

2017-03-24T00:00:00+00:00

Yeah, I missed it. But this is fine, I don’t blame anyone for this. I slept and woke up knowing exactly what I had done. I think the repenting part is more important than just missing it. (It also means I can sit down for 1 hour and write continuously about something that I haven’t decided yet!)

Today was the Eastern Groups performance for the Social-Cultural General Championship in IIT Kharagpur. These are inter-hostel competitions which are elaborately arranged, practiced for and performed. It’s kind of a big deal for everyone involved. I have done it since my second year. (Our hostel’s performance was around 2330-2355 last night. I now know that we won the Gold medal for our performance!)

LISTEN TO THE LAST SONG!

Anyway, after this I got back and didn’t write, even though I planned to. The topic that I planned to write about was Internet Shutdowns. A little bit of backstory on how I got to know about this particular topic was this On Air With AIB - Broad Bandh episode. They talk about how the Internet was shutdown in India by state governments at different points in time for absolutely arbitrary reasons. That they did was a surprise to me because I have never been in a state where the government shut down the internet, and these things also don’t happen to make it to Popular media (or print media), both of which are SURPRISING AS HELL. Internet shutdowns are a big deal and I would like to know that state governments can do it.

After seeing this, I was interested in how exactly they do it. Asking ISPs to switch off their routers is one extremely simple way to do this, whether they would or not, if they have legal recourse, etc etc are topics that I don’t know much about and won’t speak because I think that ISPs would rather stay in the good books of the state government considering that they need the state government’s cooperation for licenses, laying infrastructure and while autonomy and independence are great ideals; to imaging that ISPs have it would be too idealistic a scenario. I can’t believe they do, they might simply just switch their routers off. (Also, from a market point of view, if the government forces ALL ISPs to switch off their routers, they are not going to lose customers because no one was able to provide internet anyway. That is an INCREDIBLY crude estimation using my limited 2nd year HS20001 Economics course and what I learnt there on Supply and Demand and how the ideal rational consumer would behave)

This graph curiously shows that there were a negligible number of Fixed-Line internet shutdowns. Very interesting! In a primarily mobile user (78%) market like India, this is exactly the way to go.

A lot has been written about this before the AIB episode. It’s an epidemic, and clearly the state governments are using at the earliest sign of trouble.

I think that the chaos that the lack of Internet propagates is much much larger than the rumour spreading nonsense that happens on WhatsApp, primarily because people forwarding messages that are cleverly constructed don’t stop when they can clearly see that the authenticity can’t be verified. Beyond that, in most cases the origin of these “forwarded messages” isn’t even clear. It goes through many people before landing in some group of people who went to the same school decades ago. Are these groups required? Maybe. Should you forward anything to this group? HELL NO.

(I will stop before this becomes a rant on forwarded messages in WhatsApp and why I think the fact that the most time that you can snooze someone on WhatsApp is 1 year is a cute effort on WhatsApp part to let people redeem themselves)

But the Bangalore story has really shown the right way to use services like WhatsApp in an IM friendly, always online world. If everyone takes this up, and squashes rumours as they are created, there’s clearly no real way for rumour-mongers to get their feet into the ground and gain traction. The authenticity of a message is easily checked by also hosting the SAME message on a state government held HTTPS domain. (Phishing these domains and creating other domains with almost identical names with an e substituted for an a, etc is extremely simple. So, you should always check HTTPS certificates and read out the name’s spelling so that your brain doesn’t trick you into believing that they are the same word, JUST mis-spelled)

Once the update system is solid enough to squash rumours and rather spread messages (also hosted on HTTPS government held domains), information will no longer be a commodity that needs to come from unverified sources.

This topic is much more vast than I previously estimated. And as a matter of fact, this is the first time that I have MORE content for this blog than I really intend to write. The remaining part of this will be about how an entire country can be taken off the Internet. Something that Saudi Arabia does often and Egypt did once. Egypt’s shutdown was particularly well documented with some articles offering a lot of technical details. I will read all of that, including the Border Gateway Protocol (BGP) and one day soon write a post about that.

Some helpful links if you are in India and would like to know more about the internet shutdowns:

POST #35 is OVER

Day 34 - Prelude to Internet shutdowns in India; Surprise documented!

2017-03-23T00:00:00+00:00

Content is needed. … The beast has to be fed

That excerpt is from On Air with AIB, the first off script segment in season 2. It’s completely true!

On the happy side, I am 78% through with Handmaid’s Tale! The story has spiraled completely off the charts. The backstory is finally pieced together now, and the present has taken center stage with some INCREDIBLE stuff happening. The commander’s character had a surprising amount of depth, and when they finally revealed (was it obvious?) that he was in one of the top positions because of which all these privileges were accorded to him, I was rather surprised that he was so different about something that he worked actively for. In fact, he even defends it at one point in time. As if he meant to say that the old times were not better but in fact worse than the present. Offred doesn’t buy it, I am sure of that, she’s so clever. Also, the daughter’s name hasn’t yet come up and the daughter is … Okay, that would be a spoiler. I don’t want to tag this. It’s fine, that’s a minor detail.

Instagram’s stories has this feature where once you post something to your story, you can actually see who all viewed your story! The last time I posted something, I didn’t notice that, but yesterday I did. It’s rather radical! (I wish it had the selfie-and-respond property that was there in the Beme app. Typing something in between a story was clunky, unnecessary and completely restrictive while watching a story.

An old song that was overshadowed by Hello when it released. It also happens to take some more space on the vlog. You should probably play that song, no matter if you have heard it before or not.

The other surprising thing that I saw / read / understood today was that Governments can shut down Internet for all the people in the whole country, or in specific parts of the country at will. Apparently, state governments in India often do this. This again was featured in the On Air with AIB season 2 episode “Broad Bandh”. I was surprised because I have never been in a place, somewhere in India where this happened. There were a lot of things that happened in Tamil Nadu in December 2016. A lot of it was rather sensitive and there were plenty of rumours making the rounds. I would have thought at the peak of all this activity, the government must ATLEAST have tried to get it done. I didn’t hear about it at all, so it is safe to assume that this measure wasn’t considered by the TN State Government.

That said, there have already been incidents in Gujarat and Jammu and Kashmir. How they do it is an interesting networking problem. This Gigaom article has a great explanation about what it means to “shut down” the Internet and how countries typically do it. (Note that Satellite Internet survices should still work even if Broadband (wired) and Mobile networks don’t work. Maybe you should all get a satellite internet WiFi Router, my bet is that they are incredibly costly and extremely slow. (Is Slow Internet worse than no Internet?: I happen to disagree. Even if connectivity is slow, you can only go to TXT file websites and atleast read at length. (Also, Amazon purchases are synchronised over Wifi and those files are very very small. For reference, a 300 page book took about 1-1.4 MB of data to download everything for me to get started reading the book. (It’s the last part: the convenience of having your book and not having to 1. go to a store, explain everything to a myrid number of attendants, and them having to tell you that they have everything in the world BUT the 10 books that you are looking for. 2. Not having to wait for the book to be delivered somewhere in Rural India.

There are LOTS and LOTS of … I lost my train of thought. I am very very tired from today. Okay.

POST #34 is OVER

Day 33 - Arduino LED Blinked for the first time;

2017-03-22T00:00:00+00:00

YES! I think that everyone reading this is going to think 1 of 2 things:

(a) A fourth year engineering student is getting so excited about getting an LED to blink on an Arduino? Are you kidding me? AMATEUR!

(b) What is the big deal?

Well, a back story is in order. I have been a part of Autonomous Underwater Vehicle since February 2014. That was the second semester of my first year, since then I have seen people around me in the Embedded and Electronics teams of this research group do stuff with Arduino, other micro controllers, small computers like Raspberry Pi or BeagleBone Black or Intel Galileo. I never really meddled with any of these and stayed within the confines of larger computers whose motherboards were out of sight.

Obviously, through the years I have had the chance to look at others writing code for an Arduino and the whole process of reset-burn-start for “sketches” written for it. So, I am familiar with the process, but I hadn’t done it myself till today. And that is why this is so important to me. To finally understand the final part in most software-hardware interactions. The micro-controller is as close to the actual circuitry as I can get for now.

By the way, I didn’t do this LED blinking thing simply because I wanted to. It is part of the process of running a stepper motor (which DID work!) using an Arduino and a Rhino Stepper Motor Driver. The connections were actually rather easy to understand, and alongwith a couple of other Masters students, we were able to get it to work today itself! It was pretty satisfying.

I am still inching ahead with Handmaid’s Tale, and the book has become rather easy to read and understand because finally the dust is settling and we are getting more information per page! I would be lying if I said that I didn’t have fun reading the first obscure and opaque part of the book.

Tomorrow will be some more working with the Arduino and other hardware to get a machine ready to work.

Meanwhile, I got to thinking about 360 cameras! What a GREAT thing, essentially, once they become the norm instead of a novelty (think: Digital Cameras in 2001) there would literally be no requirement of a person to photograph or videotape small family events like birthdays. There would be no Camera to “look into” and say “cheese”. Everything around the camera upto a particular distance IS the frame of the camera! No one is out of frame, you don’t have to squeeze anymore, In fact, a circular group photo around the camera would make a lot more sense because it could then be flattened and posted anywhere! So, they are very very exciting for the casual video taping of events. Also, it makes the experience easier because there is no longer someone who needs to monitor the camera. I feel like I already said that and simply rephrased and wrote it again. Ah, whatever. This series is not about quality writing (GOD NO!), it’s about process. (And I am starting to enjoy it.)

P.S. Today is the 1/3rd milestone. Slightly less than that. (33.33333333…)

POST #33 is OVER

Day 32 - Game of Thrones Questionnaire; Bias for our solutions; awesome-social-science

2017-03-21T00:00:00+00:00

Today, I watched this video. That video linked to The Game of Thrones Research Project which is a group of people who have put out a public questinnaire and are keeping the responses open for a period of time. They plan to do something with the results and entries that they are going to get from it. Apparently, this kind of thing falls into the genre of Audience Projects.

There is a longer explanation of what they plan to do with the data, and what they hope to learn from it on this page, and theses goals are mentioned there:

(1) To capture the range of responses that people have to the series, overall – why and how it matters to people to follow the story, and see how it unfolds, but also to understand why and how some people have been upset by particular elements. (2) To see how, for different people, this series finds its place among other modern fantasy stories. (3) To understand how people are bonding with the very different kinds of characters that GoT offers us, and who so often have awful fates ahead of them – and what sorts of pleasure you get from this. (4) To find out how people relate to the big controversies that have accompanied GoT – for example, the controversies around the changes from books to TV, the altered fates of various characters, and the series’ portrayals of sex, and sexual violence. (5) And to have a go at thinking how this ‘fantasy’ world of Westeros and beyond is seen to relate to our world, our mucky, violent, troubled world.

Now, the third point above about the characters is especially interesting to me! I would absolutely LOVE to know who the most liked character is (among the show fans, this might unequivocally be Tyrion Lannister, but among book audiences, I want to believe that Jaime has a chance. Time will tell.

Today, I also worked on this issue. The main goal of this issue is to create a “search” URL parameter which can be pre-filled into the only search bar on the MFQP frontpage. This would allow for linking directly when sharing with other people or when adding to course pages on the Wiki itself. (That way whenever new papers are added, they will automatically show up in the MFQP link that is available on each course page)

There was some back and forth about the approach to solve this problem. The contributor, Punit implemented this by entering into the message passing part of the code and editing it. Thinking about it at first, I believed that it would be a really small change, and a bit more investigation led me to this patch, which uses the jquery-url library and fills the search box and then calls the search() function which will read the query, call the fuzzy search library that is integrated into MFQP and finally show the results. This works!

My main arguments for my way of solving this problem are 1. the size of the patch is way smaller and 2. it re-uses all the existing code and simply adds some basic JS to use that code instead of delving into the message passing mechanism. Also, my first impressions on seeing the old code (which I was doing to get context on Punit’s pull request) was that the message passing was being used for the sole purpose of communicating that the data.json Network request has been completed and if there is any query that needs to be made, it should be made right now.

Since the discussion wasn’t going anywhere with just the two of us, I called for some more eyes on the code. The bias that we have for our own method of solving a problem and believing that it is the most elegant and pretty way to solve a problem often gets in the way and it’s best to get a second or third opinion instead of trying to re-introduce objectivity to the discussion. How emotional we are about a few lines of text that we write? :D

Also, this hilarious tweet series went out from me when I saw the first PR to my awesome-social-science list:

This was yesterday, or the day before. Dhruv Apte (@the-ethan-hunt) was kind enough to work on it again and submit another pull request, this time for the Golem Effect. I added the Pygmalion Effect entry some time ago, and it was a very very interesting read. I still remember the lighting conditions changing for factory workers and the external supervision by higher management, and how these two affected workers performance. TL; DR Higher management presence even with bad lighting yielded good performance from all the factory workers!

I look forward to reviewing that pull request and probably making some changes and merging it into the repository! One of his references came from the American Psychological Association, which surprisingly lets it’s users download full text PDFs of ALL the articles that are on the website. I was really surprised that they let people do that. Being inside IIT Kharagpur, I have access to some of the popular services like Elsevier, IEEE, ASME standards, but there are still a myriad other publications out there which charge a whopping $20-$30 for every research paper! (For comparison, 300-page Handmaid’s Tale cost me slightly less than $4.00)

Anyway, the APA website looks good and hopefully I will be able to find some time to read through some of the interesting articles. Also, it looks like a lot of D-News were sourced and fact-checked here. It would make sense too, the issues look long and can be a day job!

I have rambled on for long enough.

POST #32 is OVER

Day 31 - Handmaid's Tale story revelation at 55%; GC Score visualization prototype

2017-03-20T00:00:00+00:00

Ah, finally! FINALLY! 55% into the book, Margaret Atwood thought it appropriate to reveal the circumstances that lead to the world that Offred and Ofglen and Of* live in. It was terrifying! I think I can now see where the comparison to 1984 comes from. I can’t really talk about it here without marking it as a spoiler for everyone who plans to read this book, so I would much rather talk about it once I have read the whole book and this particular event has more context around it. That said, this is a great book, I can make that judgement on the basis of whatever I have read till now. It has been amazing! Sure, the author took a lot of time to get to the backstory, but the story that she did spend her time on was also interesting and worth reading.

Personally, I like books that keep the readers on their toes all the time by changing the setting, going into long monologues that compare a small feature of the present situation from something in the character’s backstory, essentially introducing the backstory and at the same time showing the readers what the character is thinking about, and what her mindset is. These books are pretty challenging and fun to read.

The next thing that I worked on today was the GC Visualization prototype. The status quo looks something like this:

The white space on the right side of the webpage in the above screenshot is where the stacked bar graph should have been.

The prototype that I built looks like this:

There are some major improvements over the previous page. One word: SPEED

It’s been about 2 minutes since I opened the Awaaz IIT KGP page, and the Google Visualization graph hasn’t yet loaded on Chrome 56 Linux x64. In fact, I think it is never going to load now. There are quite a few errors in the JS console.

Edit: The chart loaded mysteriously when I opened it about 15 minutes after. That’s where the third screenshot above is from.

D3.js is all about speed! It’s an incredibly small library and there are a LOT of tutorials related to every single topic in D3. In fact, the code that I have used in the second stacked bar graph above has been directly copied from a bl.ocks.org created by the author of the linked tutorials. He has a lot of other samples that he has put up on BLOCKS.

D3.js also offers two modes, HTML and SVG. The HTML mode is plain HTML made to look like Graphs! SVGs are of course related to more complex charts, but for the simple bar chart shown in the first screenshot above, I have simply used the HTML bar chart! It’s FAST!

Another logistical improvement for whoever is adding data to the website is that the data is stored in CSV files, which can be directly exported from an existing Excel sheet. Right now, the data is stored inside javascript lists like this one:

    var halls =
    [
      ["hall","Athletics","Cricket","Chess","Squash","Football",
      "Weightlifting","{role: 'annotation'}"],
      ["MS",20,10,5,0,0,0,35],
      ["RP",8,6,3,6,2,0,25],
      ["Azad",0,0,1,2,10,10,23],
      ["RK",0,4,0,0,4,6,14],
      ["LBS",12,0,0,0,0,0,12],
      ["HJB",0,0,0,10,0,0,10],
      ["Nehru",4,0,2,0,0,2,8],
      ["JCB",0,0,0,0,6,0,6],
      ["Patel",0,2,0,0,0,4,6],
      ["MMM",0,0,0,4,0,0,4]
    ];

Whereas a CSV file is far easier to handle. Adding a new event here would require adding a column at the second last place (unless this has been exported from some CSV file using another script, which is TOTALLY possible).

In any case, I have sent the prototypes out to some people. Hopefully, I can get this on the Awaaz Website as an alternative way to view the General Championship scores. This kind of a visualization improvement would make score tracking and following developments very very easy!

This being a prototype, there are definitely improvements that can be made in this. Most importantly, the colors. Right now, I have just used the six colors obtained from combinating ff around the 3 places that are available. And then using it twice. The colors are easily recognizable, but they are still really bright and that’s probably not the best way to do this. (I wish I would understand how to pick the right colors!)

Going back to the book, I hope that I can complete reading this book this week. After this, I plan to read a book by Jane Austen (see Day 20: … Jane Austen plans ….

POST #31 is OVER

Day 30 - Metakgp meetup to meet the candidates for Maintainer Election March 2017

2017-03-19T00:00:00+00:00

Today, we had a Metakgp meetup to meet all the candidates in our upcoming maintainer election. So, to be clear, we are not looking for a maintainer for one of our projects, rather we are looking for a maintainer for Metakgp. Their roles are diverse but would include day-to-day management, resolving conflicts between members, making the right decisions with inputs from everyone in the community and pushing the community towards a consensus if the community can’t find it on it’s own, and finally fascilitating an environment where new people would feel comfortable joining and contributing. This role is much better described at the Metakgp:Governance wiki page.

We have a lot of candidates this year. And the process is proceeding pretty smoothly. Everyone who has made a few contributions on the Wiki and atleast one commit on one of the repositories in the Github organisation is eligible to vote in this election.

Today’s meetup was about meeting the candidates, getting to know them, and seeing them and talking to them about their ideas and proposals for metakgp and how they plan to move the community forward during their tenure as a maintainer. Personally, I just wanted to meet all these people who are probably going to be making some major decisions in the next 4-12 months. It was a great meeting, and it ran for about 3-4 hours and we talked about all kinds stuff with each candidate in turn. (It was structured too!)

There were a lot of points discussed, I would like to touch on a few of them here.

Involvement of a professor

At metakgp, we have been discussing getting a professor who can be “in-charge” of the group for a long time. Almost ever since it was formed. The main idea for this is so that we can have someone whom we can get to sign the documents that need a professor’s signature (mainly, room booking forms) with far less friction than right now. We have been dancing around the idea for almost 2 years now, and it has never materialised until now, when we truly discussed this rather seriously, probably because we are a student run organisation right now. We don’t have any funds or anything. We contribute for server costs, we use the free plans of most of the services out there. And not having any oversight ensures that we can do whatever we want, without the fear of offending someone in the process.

Although, logistics does become harder with the lack of a professor and ends up being prohibitive when trying to organise talks or movie screenings. (Last year, Harsh Gupta and Vikrant Varma collaborated to organise a screening of the Aaron Swartz documentary)
Introduction seminar / Workshop / Hack days

In Kharagpur, each year 1400 starry eyed 17 year olds enter the campus as first year students. They are waiting to be poached by societies around the campus. To fascilitate this, Technology Students Gymkhana organises introductory seminars for all of the popular socieites like Robotics research groups, Technology Literary Society, Technology Dance Society, etc. Initially, I really liked this idea. It seemed like a great way to spread the word among a huge amount of people right when they enter KGP, and get them to start using metakgp from the get go. But a point that Sayan Sinha (@americast on Metakgp Slack) brought up was that people don’t get to learn anything at an Introductory seminar and thus it’s more or less a waste of people’s time and a lot of people might not even remember what was said there.

This got me thinking about whether this is a good idea or a bad one. There is some kind of a compromise to be made here. Clearly, there need to be a few improvements to the basic intro seminar idea (bad) to add a few ingredients from the Hack days (good) to make the final result good.

Personally, I feel that talks are a great way to spread knowledge, talk to people, and really get into the habit of sharing your experience and the things that you have learnt with others, not because they will not make the same mistakes, but because when they do, atleast they will know how to make it right!
Core groups inside Open Source projects

This is something that one of the people mentioned. I am not going to talk about it in today’s write up, I need to make sure that there’s no issue that they have with me talking about this. SO, I am going to leave it here.
Getting people to contribute (and more specifically, getting more female contributors in KGP

This has been something that was recently brought up on the Metakgp Slack #random channel. It is summarized here by Naresh R (one of the candidates in this election).

Plainly speaking, to get more people interested in a project that you are wildly interested in, but they feel lukewarm about is an uphill task. There have been a lot of magical ways to do this that have been suggested in the past, but I am pretty sure none of these really worked.

To make this happen in an environment where money is not one of the motivations to contribute and spend your time on a project is probably twice as hard (if not more). From the closely knit open source communities around prominent projects like Git, the Linux kernel, Ruby, Python, Rails, etc it is clear that even they are not able to “retain” people beyond an off hand patch that people send now and then to really understand how an organisation works (I sent one to GIT, too!)

There are also probably some specific reasons why women don’t feel comfortable working on Metakgp even though most of our work and discussion is done online on Slack and Github, both of which are open to everyone. And the meet-ups that we have happen at neutral places where both genders can roam around freely. These are justifications as to why we are not biased, but there are some other things, like a low female participation causing fewer new female members to join, which I don’t know how to solve. Most maintainers have this on their agenda for their tenure, so I hope to be able to work with them and everyone else active right now to make this happen. Progress will probably be slow, but we MUST make what little headway we can.

POST #30 is OVER

Day 29 - MISSED; Waiting for The Winds of Winter (GRRM);

2017-03-18T00:00:00+00:00

I have made very very less progress with Handmaid’s tale. It’s not that the book is not good, it is Good, but it’s confusing and highly taxing to read. The story keeps shifting into either a flashback, or an abstract comparison with something from Offred’s previous life, and it gets rather hard to keep track if you are travelling or something. I hope to make a lot more progress in the next few hours. I am going to install Abaqus (Ansys is installed, but Abaqus apparently is something that people are more familiar with), so that should give me a few hours of solid reading.

This post was again missed, I keep thinking about whether I should start using beeminder.com just to ensure that I have a stronger motivation to wake up and write! As a matter of fact, I slept early last night with intentions of getting back up again and writing (NEVER works out) and the first thing that I thought of the moment I woke up this morning was that I had missed posting last night! That shows motivation right?

Besides all that, there’s been the huge new Wild Cards deal, the Wild Cards reread, lots of stuff with HBO that I cannot talk about yet, and of course – always, always – WINDS OF WINTER.

So much to do. And the days go by so quickly. I love my work, all my work, all my projects, all my children, but sometimes it seems as if the harder I work, the further behind I get.

Maybe things will calm down soon. But I am not holding my breath.

From GRRM’s blog post. The Winds of Winter has been due ever since last March. I completed reading the 5th book, A Dance with Dragons in December 2015. It was the completion of a series of book in which each book had about 1000 pages. There have been rumours ever since that the 6th book will be released within the next three months. Yeah, constantly. There were a few rumours about it coming out before Season 6, then the book coming out in December 2016, and now, this. There’s almost no hope that the book might come out this year (?). The most pessimistic predictions for TWoW were that it would come out in 2019, these predictions were from people who had waited for the fourth book for 5 years (from August 2000 to October 2005) and then 6 years for the fifth book (July 2011). 8 more years for the 6th book? Frankly, I don’t know who could have predicted that. It’s been 5.5 years already, anything before 2019 would be a good thing, I guess from here on out.

I did most of the work for an assignment that is due next week. That was a lot of C and a lot of matrix copying / extending / re-computing. Rather fun!

POST #29 is OVER

Day 28 - Short post; Nothing here, move along

2017-03-17T00:00:00+00:00

I don’t really know what to write about. It’s been about 15 minutes since I opened this file and edited the header. No idea has really occurred to me, yet.

The next issue on Github that I am going to target is the subjects.json inconsistency that is bound to happen between all Metakgp repositories.

Basically, subjects.json is meant to be a JSON file that maps subject courses offered in IIT Kharagpur to the name of the subject. Right now, there’s a JSON file that’s used. I can’t be sure that the same version of the file is used in these projects: mfqp-source, gyft. These are the only two projects I can find which use this file, I was convinced before that more projects used it! Probably because people have often asked me for that file, and used it to some other end.

Anyway, submodules would solve this problem entirely. Must work on that.

Still struggling to find something to write words about

I am going to have to end this here, it seems. This will definitely be my shortest post till now. I will strive to do better tomorrow!

POST #28 is OVER

Day 27 - MISSED; Some python and regex written for Gyft

2017-03-16T00:00:00+00:00

ARGH! I missed yesterday. It wasn’t even too tiring a day or anything, anyway, I am going to make the best of this. I am about 8 hours later than when the usual post should have been.

I have not been writing a lot of code, and I hadn’t submitted a pull request in almost 20 days! That’s unacceptable, Jimmy, UNACCEPTABLE.

I decided to remedy that situation today. Gyft is a set of python scripts which lets you login into your IIT Kharagpur ERP account, get your timetable for the semester from there. Before my patch, the only option was to put this timetable into your Google Calendar. You generate a client_secret file and give the python script access to add recurring events to your calendar. It didn’t sound right to me, and it was pretty cumbersome to use as well.

So, I wanted to enable exporting the calendar to ICS files. ICS files are amazing! Every major calendar application supports them, they are apt for static calendars like a Semester timetable. I had this idea in August 2016, I opened the issue in December 2016. Since, it has just been vegetating because I wasn’t working on it, and the whole Gyft repository has been rather inactive since.

The first part of the patch that I had implemented already was the relatively time-consuming part. It had a function which would build an event given the appropriate parameters like start time, end time, etc.

The second part of this patch was to merge this with the parsing code that is there inside the add_events.py script. It takes the data from the data.txt file (which is taken from the ERP) and parses that into event information.

Here comes the Regular Expressions part. The old parsing was done with a complicated string parsing mechanism. Frankly, I couldn’t understand what was being done, and I took a step back and analysed the string that we do want to parse. It looks something like this:

10:0:AM-10:55:AM

I realised that this was the best place to use Regular Expressions! A couple of minutes in the Python shell led me to this regular expression:

TIMETABLE_DICT_RE ='([0-9]{1,2}):([0-9]{1,2}):([AP])M-([0-9]{1,2}):([0-9]{1,2}):([AP])M'

This Regex is clearly not optimal, there is no need to repeat the same Regular Expression twice. Having it only once, should be enough. I should implement that!

But barring that, it worked out pretty well. The new parsing code which is part of the new generate_ics.py script was smaller and also returned a datetime object instead of trying to write the complete iCalendar event as per the RFC (IETF RFC 5545).

Eventually, this pull request came out of my work today! I am awaiting reviews on the pull request, and it should get merged soon. I have already generated an ICS and added it to my Google Calendar! It looks good!

POST #27 is OVER

Day 26 - Communication on Slack;

2017-03-15T00:00:00+00:00

Today, I wrote this

START REPOST

Other important things I learnt along the way

Listen to others, think before you start typing

Slack and IM make communication easy … and complicated. Let me explain:

Communication is easy if the parties communicating have a common goal, and are putting in a non-trivial amount of effort into keeping the conversation sane, unblocking themselves and getting to that common goal.
Communication is easy if the parties communicating have conflicting goals, but listen to the other party and are willing to change their stance if reasonable, technically sound arguments are posed.
Communication is easy if the parties communicating have opposite goals, are unwilling to change their stance, but would still comply with someone else’s opinion because of their inherent belief that the other person is more experienced than them.
Communication is complicated if one of the parties communicating is responding without reading and understanding the points being made in a discussion.
Communication is complicated if one of the parties doesn’t know enough to comment about something but still chooses to do so. See Dunning Kruger Effect
Communication is complicated if an inexperienced party pretends to know how often something might happen, despite the experienced party clearly explaining the reasons for their claim. (See Dunning Kruger Effect)
Communication is complicated if one of the parties is not interested in reading the points that are being made or have been made in the past by the other parties and their only interest is to propagate their own point of view, because of reasons such as: ease, naivete, lack of the required experience

If a solution approach seems complicated, Don’t shy away from it. Embrace it.

Does a proposed solution to a given problem sound very complex and extremely time-consuming? Don’t shy away from it. Try to get it done. Really try, spend your time (and money) doing it.

Most of the times, you will solve the problem. Or you will come really close to it! That’s good enough for the first time you do something complicated.

But a few of the times, things won’t work out. You ended up totally wasting your time. But is it really a waste of time?

You just discovered a very complex solution to a problem. If it is a common problem, why isn’t there an easier solution? Investigate, solve, spread the word. You just made life easier for a lot of people.

And in all the other cases, you just learned that something was very complicated. So complicated that you can probably share your experience about doing it, once you manage to crack it. That’s one hell of a story to tell everyone!

END REPOST

Okay, story time! About the second part. I can speak about it confidently because I have had it happen to me once, and I chose to embrace the approach and everything worked out fine in the beginning.

There was this one time, where I wrote a Rails application. It was called the mentorship portal. It is a portal to facilitate communication between Students and Alumni through the Student Alumni Mentorship Programme. I chose to write it in Ruby on Rails with help from Rahul Mishra. It was a pretty big Rails application and I was soon losing track of how the development was going, I learnt a lot more about Rails. But alas, deployment is where it really takes the centre stage.

The most famous method of deploying Rails on a LAMP stack seemed to be with Phusion Passenger. And thus, it began. You might want to let this song play in the background while you read the account. I have found that this song makes a lot of accounts a lot more fun to read because it is fast paced, and nostalgia wells up:

I installed Ruby and Rails on the server. Apache was already installed, I knew nothing about reverse proxying or apache configuration files till now. I knew that you had to define the root of your websites in the apache configuration files. That was the extent of my knowledge.

I was following a much older (and uglier) version of this doc on their website. The present documentation of this process seems to be really nice!

Installation went through smoothly. Though there are two challenges here that I would like to mention: The server didn’t have “full” internet, it is behind a HTTP proxy server. The server doesn’t have any mailing access, there are no other mail relays which have been configured to be used by default.

The most important step till now, is the addition of the VirtualHost tag to the appropriate configuration file. There was a bit of a struggle trying to find out exactly which one I wanted to add the code to. And I struggled a little bit more to understand what exactly would be the “replacements” that I would have to make to make the piece of code that’s there on their website to work.

The problem with replacements is that they don’t cover corner cases, like ours. We had a subdomain which had to be prefixed with www to be access from outside the institute network, and not prefixed with it to work from inside the campus. I see that the documentation still doesn’t cover the case on what to do for subdomains. I think we finally solved by adding a ServerAlias attribute inside the VirtualHost tag.

All in all, this must have taken about a week’s worth of 4 hours a day scratching my head about what to do, searching and going through a LOT of StackOverflow answers and reading the documentation again and again to try to find some hidden insight into what might be the problem: A did-you-know box that would magically tell me what I had missed.

(I realise now that I have made it sound a little bit too easy, perhaps! But considering that I am writing this 2 years after the fact, I am happy with the detail I could get.)

Oh, I forgot to mention that there was a blog post that I wrote then. Presenting Deploying on an in-house server (RANT) dated 1st May, 2015. I have really invested myself in ranting about deploying on in-house servers!

POST #26 is OVER

Day 25 - Quarter of the way there!

2017-03-14T00:00:00+00:00

I am 25% there. That’s one part in four, I just need to do the thing that I have already done, 3 times over. Sounds doable, from here on atleast! I just wish I don’t miss a day in between. I missed one day in between till now, and that was not even the day I was most tired on. In fact, the days that I was most tired on, turned out to be the challenging ones when I struggled to write the 500 words and git add and git commit and git push.

I would like to talk a little bit about the encrypted FS setup that I have got going on, in my computer right now. It’s been about a week (or two?) since I installed Ubuntu 16.04, and I setup two encrypted partitions when I was installing it. Now, the dust has settled for the most part, and it is looking good now! This post is a continuation of the post on day 15.

I added these two volumes to /etc/crypttab and to /etc/fstab.

/etc/crypttab

partname UUID=... none luks,nofail

Here, the luks and nofail in the last argument are VERY IMPORTANT. Without the nofail flag set in that argument, the boot fails because it couldn’t find the right password for that block device. Also, the none must be spelled out so that Ubuntu will interactively ask you for your password. This is only one part of the setup though, only adding this will also lead to boot failure. More on how to recover from there later.

/etc/fstab

/dev/mapper/partname /media/username/mountpt ext4 defaults 0 2

The partname is automatically referred once the block device has been unlocked using LUKS and there are two parts of the partition now (the decrypted one is mounted at the provided mountpoint).

Not doing any of the above steps properly, will lead to a failure in the boot process. What this means, is that the boot will simply fail and bring you to a black screen (sometimes the screen also has text, I was a little bit panicked to check what was going on on that screen).

A simple way to get out of this is to boot into Ubuntu in the recovery mode, and from there get a root shell. This root shell only has read-only access to the disk. So, from here, you should run something like reboot which will restart your computer. There is a sequence of commands that can be run here to get you to a root shell with write access to the disk. I used this sequence to reset the fstab and crypttab files to their default values when I ran into trouble. Unfortunately, I can’t remember what that sequence is. I am sorry if this part is so unhelpful.

I can only say that rebooting was definitely part of the process, and then something like “continue boot” in the recovery mode list.

Once on this write-access enabled root shell, you can do one of two things

Rollback all changes to these two files, and restart. This will definitely work all the time. You can simply make the right changes and then restart to get it to work properly again
If you are feeling adventurous, then you could change the files in this root shell and reboot. If you have made the right set of changes, the interactive password prompt will come up.

If you don’t enter anything in the interactive password prompt for a long time, then it goes to a blank screen. There’s absolutely nothing on that screen. I had to pull the plug and restart both times that I ran into this problem. No clue whether there is a better way to pull the computer out of this situation.

One thing that I have not yet tested is what happens when you enter the wrong password in the interactive prompt.

POST #25 is OVER

Day 24 - Rambling about the Ubuntu 16.04 bug a little bit more

2017-03-13T00:00:00+00:00

I didn’t watch Revenant, actually. In the day 21 post, I said I would watch it, but I couldn’t watch it. I started watching it, but they weren’t getting to the point. I had a similar reaction to Avatar when I saw it for the first time, I loved it afterwards. So, I will try again some time soon.

I started the day with this tweet which I thought indicated the end of my troubles with the left click on the mouse not working with a new Ubuntu 16.04 LTS installation on a Dell laptop. It was not, I faced the problem again about 12 hours after the tweet, and I tried this but the problem didn’t fix itself. Interestingly, on disabling the ID that the DLL device shows, it simply re-appears with a new ID. This happened earlier also, but I thought this was somehow the fix. I tried disabling the complete pointer and that didn’t work either.

This bug has been reported before with Ubuntu 9.10 (yeah, that’s about 8 years ago) on Launchpad. The only way to reproduce this bug that is mentioned on this bug page is to “Click inside a Flash screen”. From the lengthy discussion on the bug there, it is clear that this is most probably a window manager issue. Changing window managers is rather easy on Ubuntu, you install something and it shows up in the sessions and you change, it’s not that big a deal, to be honest. I would change as soon I find something that I like. I have tried Tiling WMs like i3 in the past, and they didn’t work out for me. They were just too weird and completely radical compared to everything that I have become used to. Maybe some other window manager! I have found this webpage dedicated to offering users choice! Just what I was looking for.

I read quite a bit of Handmaid’s Tale yesterday. I am about 1/3rd of my way through that book, it is disturbing. Deeply disturbing. I would not compare it to 1984, because 1984 was about setting the scene as soon as possible and explaining the character’s dilemmas in that strict system. This book is more about a character who has accepted her situation, and slowly revealing the situation that she is in, and the life that she had before she walked into this one. I have read the synopsis so I know a little bit too much, I wonder how it must have felt to have read the book without reading the synopsis, which decidedly gives away a lot. (It would have taken atleast 45 pages for a reader who hasn’t read the synopsis to understand what was going on with all the women wearing red, whose names all start with “Of” WHY?)

POST #24 is OVER

Siddharth Kannan's Blog

OpenCode is Remarkable

(What) Is This An Ad (For)?

Post 10 - Inflation-adjusted Rate of Return

Post 9 - Metabase and Sankey Diagrams

Post 8 - Civilization 6 (Game)

Post 7 - Using MacOS

What Did Not Work

Post 6 - Short Term vs. Long Term

Post 5 - Emacs and Elisp

Post 4 - Science Fiction and Time

Post 3 - Yet Another Streaming Service

Post 2 - Games and Movies - Violence

Post 1 - Installing Linux is (Somewhat) Easier Than It Used to Be

Updates to the NVRAM

Grub, kernel, and initrd inside /boot

Updates to /etc/crypttab and /etc/fstab

References

Using LLMs Daily

What Works

What Doesn’t Work

Red Lines

The Promise of Open Source

The Perfect Window Manager

Options for Tiling Window Managers

Predefined Layouts

Just Use i3, Then!

What Next?

Better Ansible Playbooks and Roles

What Do You Need?

Naming

Testing

Internet Access

Conclusion

Adventures with Wireguard and MTU

iptables

Docker

ufw

tcpdump

Other theories

Answer: Max Transmission Unit

Further research

Using an iPhone Without Windows or Apple Computers

Photos And File Transfer

Old Process: Using Windows

Plaintext

SSH

Notes and Review - White Noise (DeLillo)

Quotes

Beyond 100 Megabits Per Second

Internet in Japan

Home Network

Problems

Conclusion

Look Up

A Chronicle of Predominant Conversations (2023)

Review - God, Human, Animal, Machine (O'Gieblyn)

History of Ideas

Metaphors

Hype

Empiricism and Science

Book of Job

Before the Cartesian Partition

Review - Subprime Attention Crisis (Hwang)

What is Programmatic Advertising?

Standardizing the Marketplace

How is Programmatic Advertising Different?

Similarities to Finance

A Bubble in The Attention Economy

What Next?

New Wireguard Peer in 10 Minutes

Conversion from HTML to ePub Format

Review - Darkness at Noon (Koestler)

Better Personal Data Management

Frequent OS Version Updates

Decouple Data from the OS

Encryption

Don’t Forget How to Install Operating Systems

Understand the Boot Process

Manage Your Data Yourself

Updates to `/etc/crypttab` and `/etc/fstab`

`iptables`

`ufw`

`tcpdump`