Archived Posts on The Learning Loom

Market Capitalization

Thu, 10 Apr 2025 20:41:21 +0530

I was browsing some Bitbucket API documentation somewhere when I read something like - Atlassian IPO’d with a market capitalization of 4.37 Billion

I wanted to make sense of that sentence a bit more than I already did.

Obvious - but worth stating for the record: Atlassian builds Bitbucket.

Atlassian was publicly listed on NASDAQ on December 10, 2015.

The symbol for the company was the memorable word - TEAM

And just in case you wish to know - NASDAQ stands for:

National Association of Securities Dealers Automated Quotations

That was a serious WTF moment for me.

Anyway - turns out I don’t know anything about:

Atlassian
NASDAQ
Or market
Or capitalization

But I do know one thing: 4,370,000,000 – That’s a huge number

Here’s a visualization I drew for myself:

Then I googled for TEAM, and found its stock market history.

I went to the very beginning, Day 1. And saw this:

So I figure 28 * 17M should give the market capitalization - but then no. That’s an order of magnitude lesser than expected.

Turns out - that is merely the trading volume for the day; that is N number of shares were outstanding - up for the grabs - and only a percentage of that ended up getting traded:

Tried trawling for data elsewhere, different sites, different places, stil not getting the numbers I need.

Finally - in the NASDAQ site saw some relevant data:

src

Tried pluggin and chugging a few numbers - no luck.

Finally I went to the source at SEC filing and found the following:

1

52,872,107 + 155,803,022 = 208,675,129

So that’s the number of shares available - making up of Class A and Class B shares.

I didn’t go into much details on this differrentiation - but there’s voting differences, etc - which is beside the point now. Many of the data trawling sites seem to get this wrong.

Anyway - now that I know the necessary details, I could calculate market capitalization:

1
2
3


market capitalization = total number of outstanding shares x share price at close of day

market capitalization = 208,675,129 * 28 = 5,842,903,612

Now - that’s 5.8 Billion. And that matches the news:

Betting > Believing

Wed, 26 Mar 2025 20:41:21 +0530

The Power of Betting in Decision-Making

Navigating Uncertainty

In real-world decision-making, uncertainty is a constant. Whether it’s a project deadline, a hiring choice, or a career move, information is often incomplete or unclear. Waiting for perfect clarity isn’t always an option—time runs out, resources dwindle, and complexity can overwhelm.

Many rely on believing—trusting intuition, assumptions, or the opinions of others—to guide them. While this approach can provide an initial sense of direction, it falls short when the stakes are high or the context is unique. A more effective strategy is betting: making calculated decisions based on evidence and outcomes.

A Lesson in Misjudgment

Once, I was deep into a project with a looming deadline. I did some quick mental math and convinced myself, “It’s tight, but we’ve got this.” Riding a wave of optimism, I skimmed progress updates and reassured myself that things were on track.

Then the deadline arrived—and the project was nowhere near completion. The issue wasn’t my team’s performance; it was my own failure to verify assumptions against reality. I had believed instead of inspected, and it cost us dearly.

Why Believing Falls Short

Believing feels natural. It’s tempting to trust gut feelings, conventional wisdom, or someone else’s word. However, this method has critical weaknesses:

Untested Assumptions: Unverified beliefs create blind spots.
High Stakes: Relying on trust alone can magnify risks in crucial moments.
Unique Contexts: Standard solutions often don’t fit specific challenges.

Examples highlight these pitfalls:

Hiring: A candidate’s resume may shine, but without a test project, a poor fit can slip through.
Projects: Optimistic projections can obscure real progress if milestones aren’t tracked.
Career Moves: A role might seem perfect, but the reality can be vastly different.

Believing can offer direction, but it falters under scrutiny. A more concrete approach is needed.

How Betting Takes the Lead

Betting isn’t about reckless gambles; it’s about measured risks and tangible outcomes. It acknowledges uncertainty and transforms it into an advantage. Here’s why it works:

Commitment Sharpens Focus: Investing resources—time, effort, or money—forces clarity.
Results Teach: Every bet provides feedback, refining future decisions.
Action Trumps Hesitation: Betting prevents paralysis and promotes forward movement.
Reality Rules: Measurable outcomes ground decisions in fact, cutting through bias.

Practical applications demonstrate its effectiveness:

Hiring: Assign a trial project. Success justifies hiring; failure limits risk.
Projects: Set a milestone—like a prototype in two weeks. Hit it and proceed; miss it and reassess.
Career Moves: Test a new role through a short contract. If it fits, commit; if not, pivot with minimal loss.

Betting converts vague hopes into structured experiments, yielding clarity through action.

A Historical Perspective: Pascal’s Wager

This mindset isn’t new. The 17th-century philosopher Blaise Pascal framed belief in God as a bet:

Believe, and God exists: Immense reward.
Believe, and God doesn’t: Minor loss.
Don’t believe, and God exists: Major loss.
Don’t believe, and God doesn’t: Small gain.

Pascal argued that betting on belief made sense—the potential upside outweighed the downside. This logic extends beyond faith: weigh potential gains and losses, then act decisively under uncertainty.

Using Bets to Overcome Deadlocks

Decisions rarely come with full certainty, yet action is unavoidable. Believing may inspire ideas, but it fails when precision matters. Betting—acting with intent, measuring results, and adapting—handles uncertainty far better. It’s a tool for progress, turning ambiguity into opportunity.

When teams or leaders face decision paralysis, betting can break the deadlock. It’s not about reckless risk-taking; it’s about structured progress:

For Leaders:
- Demonstrates confidence by committing to a direction.
- Cuts through hesitation with thoughtful choices.
- Example: Setting a bold goal to align and energize a team.
- Encourages responsibility and decisiveness.
For Teams:
- Moves from debate to action.
- Provides clear targets and accountability.
- Fosters learning—even from failure.
- Keeps momentum going.
In Uncertainty:
- Selects a single option when too many exist.
- Tests small ideas before full commitment.
- Turns confusion into growth opportunities.

Betting isn’t about perfection—it’s about progress. It helps leaders make decisive choices, strengthens team cohesion, and prevents stagnation. The next time you’re stuck, try placing a bet: define a goal, commit to an action, and learn from the outcome.

It may just work better than endless debates about “beliefs.”

I Disagree With Jeff Bezos

Sat, 22 Mar 2025 20:41:21 +0530

It’s not every day that I find myself at odds with someone like Jeff Bezos (on a matter of principle, of course).

The man who built Amazon into a global phenomenon once said, “Good intentions don’t work. Mechanisms do.”

At first glance, it’s a compelling statement—practical, grounded, and appealing to the results-driven crowd in business and economic worlds.

But the more I reflect on it, the more I disagree.

I will argue here that intentions, far from being irrelevant, are foundational to improving the quality of human existence.

My reasons - in short - are as follows:

Intent is about us - humans; effect is about the world at large.
Intent is spiritual or psychological in nature; effect is economic or physical in nature (usually).
Intent is often a relatively long-term phenomenon; mechanism is usually more short-term.

A bit of history…

Look at the sages, prophets, and spiritual traditions that have guided humanity for millennia.

From Buddhist monks to the wise teachers of ancient India, they’ve all emphasized one thing: get your intent right.

Why?

Because intent is tightly coupled to the quality of your consciousness - which, in the long run, determines actions that lead to the individual and the world’s future.

Contrast this with the economic perspective, where intentions are dismissed as fluff.

Economists and business people often argue that what matters are effects—tangible results, second-order consequences, and measurable outputs.

And mechanisms, they say, are what drive success.

But I think this view misses something fundamental.

Putting the idea through the “common sense” filter

Let’s test this idea with a simple scenario.

Imagine you’re facing a brain tumor, and you need a neurosurgeon.

You have two options: one is a brilliant technician with a criminal background, whose intentions you distrust; the other is less technical but genuinely committed to your well-being.

Who do you choose?

Most of us, I suspect, would pick the latter.

Why?

Because we instinctively know - intent matters.

We instinctively sense that someone who doesn’t have our best interests at heart—no matter how skilled—can’t be trusted with something as precious as our life.

Mechanisms alone don’t cut it when the stakes are personal.

Time Horizon

Now, let’s widen the lens.

Consider the longest-standing organizations in history: churches, temples, spiritual lineages, educational institutions, and service-oriented groups.

These entities have endured for decades, centuries, even millennia—often thriving regardless of economic conditions.

Take the Shankaracharya tradition in India, a lineage of wisdom that has persisted for over a thousand years.

It’s not built on economic principles, which are inherently dynamic and unstable, but on an unchanging commitment to truth and human upliftment.

What do these organizations have in common?

They operate on what I call the “Intent Principle” rather than the “Effect Principle.”

Their focus isn’t solely on short-term outputs or market fluctuations; it’s on a deeper, enduring purpose.

And that purpose—rooted in goodwill and noble intent—carries them through storms that topple profit-driven systems.

Maybe Intent In Itself Can Be Seen As Part of Larger Mechanism

Of course, I’m not saying mechanisms don’t matter.

A well-designed system can amplify the impact of good intentions.

But intent isn’t separate from the mechanism—it’s part of it, especially in human organizations.

When we sense that someone doesn’t wish us well or lacks consideration, we pull away.

That’s the genesis of disorder, distrust, and collapse.

Conversely, when intent is genuine and inspiring, it binds people together, fueling cooperation and resilience.

Intent is the foundation; mechanisms are the structure built upon it.

The Durability of Intent

Economic systems, by their nature, are concrete but fleeting.

They rise and fall with markets, technologies, and trends.

Intent, on the other hand, is abstract yet enduring.

A noble thought—a commitment to benefit others, to serve a country, or to enhance well-being—can survive generations.

It’s not easily destroyed, even under adverse circumstances.

History shows this: the goodwill and aspirations of our ancestors continue to inspire us long after their physical creations have crumbled.

Think about it: no truly noble thought ever dies.

It plants seeds that bloom into future actions, often in ways we can’t predict.

Mechanisms, being rigid and material, eventually break down.

Intent, being fluid and timeless, adapts and persists.

A Life Built on Intent

So, where does this leave us?

I believe good intentions matter more than mechanisms—not because they’re a substitute for results, but because they’re the firm ground on which all lasting results rest.

A steadfast commitment to an inspiring intent is the best way to make the most of our lives, both for ourselves and for others.

It’s what drives us and our associates through decades, even lifetimes.

It’s what survives beyond us.

Jeff Bezos might disagree, and the economists might scoff.

But I’d rather take my cues from the sages who’ve stood the test of time.

Get your intent right, and the mechanisms will follow.

That’s the real power of intent.

Organizations That Last

Fri, 21 Mar 2025 20:41:21 +0530

Some organizations stand the test of time, lasting centuries or even millennia.

They don’t chase rapid growth, which often leads to collapse.

Think of a company like WeWork,
that expanded too fast and fell apart due to mismanagement.

In contrast, some of these enduring groups focus on purpose, stability, and community.

For example, small family-owned businesses often endure because they prioritize legacy over profit.

They grow slowly, staying rooted in their communities, and pass their work through generations.

Take Kongo Gumi in Japan: founded in 578 AD, this construction firm built Buddhist temples for over 1,400 years across 40 generations before being absorbed in 2006.

Or consider Antinori, an Italian wine producer since 1385, now in its 26th generation, thriving by blending tradition with modern techniques.

Then there are monasteries and convents, built on spiritual missions that don’t bend to trends.

They’re often self-sufficient, growing their own food or making goods like wine, and their isolation keeps them steady.

The Monastery of Saint Catherine in Egypt, founded in 565 AD, has survived invasions thanks to its remote spot at the foot of Mount Sinai.

Similarly, the Abbey of Monte Cassino in Italy, started in 529 AD by Saint Benedict, has been rebuilt after destruction—like during WWII—because its purpose endures.

Educational institutions also stand out for their longevity.

They meet a core human need: the pursuit of knowledge.

With strong governance, endowments, and alumni support, they adapt while staying true to their mission.

The University of Al-Qarawiyyin in Morocco, founded in 859 AD, is the world’s oldest university, evolving from a madrasa into a modern institution.

In the UK, the University of Oxford has been teaching since 1096, balancing tradition with academic excellence to remain a global leader.

Broader religious institutions, beyond monasteries, also last, tied to spiritual and cultural roles.

They often own assets like land and have global networks for support.

The Catholic Church, established in the 1st century AD, has weathered wars and schisms with its centralized authority in the Vatican.

In India, the Advaita Vedanta tradition, founded by Adi Shankaracharya in the 8th century AD, continues to thrive.

Shankaracharya established four mathas (monastic centers) in Sringeri, Dwarka, Puri, and Joshimath to preserve his non-dualistic philosophy, and these institutions still guide spiritual seekers today, rooted in ancient texts and practices.

In Japan, the Ise Grand Shrine, a Shinto site, has been rebuilt every 20 years for over 1,300 years, a ritual that keeps its spirit alive.

Guilds and trade associations offer another angle. They protect professions, set standards, and build community, adapting over time to stay relevant.

The Worshipful Company of Goldsmiths in the UK, founded in 1327, still regulates the goldsmith trade today.

The Hanseatic League, a network of merchant guilds from the 13th to 17th centuries in Europe, shaped trade practices that echo in modern organizations.

Charitable foundations endure by focusing on long-term missions, backed by endowments and stable governance.

The Rockefeller Foundation, started in 1913, has funded global health and education for over a century, tackling issues like climate change.

The Wellcome Trust in the UK, since 1936, uses its £30 billion endowment to support medical research, including the Human Genome Project.

Finally, indigenous and tribal organizations show remarkable resilience, rooted in culture and land.

The Iroquois Confederacy in North America, formed around 1142 AD, unites six nations and still operates today, its governance even influencing the U.S. Constitution.

The San People of Southern Africa have maintained their communal structures for thousands of years, passing traditions through oral storytelling.

What ties these organizations together? A clear mission, strong community ties, and a focus on sustainability over reckless growth.

They prove that lasting impact comes from purpose and patience, not chasing scale at all costs.

Structure And Meaning

Thu, 20 Mar 2025 20:41:21 +0530

“Up, down, up, down, up, down, …”

What comes next?

You know the answer - it is “Up”

How did you know it?

Your mind inferred some sort of structure, and made a guess

I mean - we do not know - what truly will come next.

But we are pretty sure - due to the repetition - and we expect it to be “up”.

There could be a breakage of pattern, but the fact that we expected an outcome - that hints at underlying structure.

Look at your favorite music

There is a structure to that - and one almost waits for one’s favorite parts of the music. At least I do.

So - our feeling of “familiarity” hints at what makes things meaningful to us.

Consider something like the following:

1
2


"asfdklj;2341p0vmlaksfjdalksdjf;la;sdjfalksdjdfklaj;sdfkja;sdkfa
aslfja;sdkfja;sldkfj;kafa;sdflkaksjdf;lakj23049i2034233490--0-0=---

Something random like that - doesn’t hold our attention much.

You skip through such stuff.

Why? Because we cannot figure out - “What does it mean?”

So the biggest point of our life is that we get a feeling of familiarity - that we get meaning.

We may be wrong sometimes as well - that’s part of the adventure but that sense of familiarity is what keeps us interested in music, story, problems.

The possibility of becoming familiar is the most alluring part of music.

Likewise, it is difficult for us to keep any interest in things that seem random, things that we cannot seem to comprehend, and things which we think we will never understand or become familiar with.

So - to be interested - we must perceive at least part of the underlying structure of the thing, and hope that you can unravel more, test yourself moe.

That’s important.

I am building organiazations, such as Hexmos and software tools like LiveAPI

Our goal with the tools and teams that we build is - to aim to construct interesting, alluring and satisfying structures.

Another observation - I’ve had is - a person who seems not to be able to find their place in the world - feels disoriented, becomes more and more of a madman by the day.

Look at any strongly religious person - you will rarely see them depressed and dejected beyond comprehension - the reason being - their mind is structured under a scheme. There is a structured organization to their mind, so it is not falling apart.

A mind that is so sceptical - that it is unable to build any structure at all - cannot survive - because it is too unstable, too random - too free.

From a psychological, or mental level, excess freedom of “anything is possible” means - insufficient structure - leads to lack of meaning, purpose and a constant sense of “falling apart”.

So we want good structure - along with a sense of adventure to unravel more of it. That is what gives meaning to human life.

A Few Things About Macronutrients

Mon, 17 Mar 2025 20:41:21 +0530

In this post, I continue the exploration from the past post, where I was trying to learn a bit more about the human body from the very basics.

MacroNutrients, Et Cetera

First, we take in food - as solids/liquids.

And that food is broken down into different macronutrients (and other components):

proteins: body-building nutrients, found in meat, eggs, beans
carbohydrates: sugars & starches, found in bread, rice, pasta
triglycerides: fats - found in oils, butter, fatty foods

Most of the generally consumed food items have different proportions of the 3 macro nutrients - proteins, carbohydrates, triglycerides.

Food	Protein (%)	Carbohydrates (%)	Triglycerides (Fats) (%)	Other (Water, Fiber, Vitamins, Minerals etc.) (%)
Bread (white)	8–9%	45–50%	3–4%	37–44%
Rice (cooked, white)	2–3%	28–30%	0.3%	67–70%
Pasta (cooked)	5–6%	25–30%	1–2%	62–69%
Meat (chicken breast, cooked)	30–32%	0%	3–4%	64–67%
Eggs	12–13%	1–2%	10–11%	74–77%
Beans (cooked, kidney beans)	8–9%	22–25%	0.5–1%	65–70%
Fruits (e.g., apple, banana)	0.5–1%	10–15%	0.1–0.5%	84–89%
Vegetables (e.g., carrots, broccoli)	1–3%	5–10%	0.1–0.5%	86–94%
Milk (whole)	3–4%	4–5%	3–4%	87–90%
Cheese (cheddar)	25%	2–3%	33–35%	37–40%
Butter	0.5%	0–1%	80–82%	17–19%
Cooking Oil (e.g., olive oil)	0%	0%	100%	0%

Some observations:

Macronutrients are only one of the many components in the food we eat
Water, fibers, vitamins and minerals also appear in significant amounts on many of our food items

Some RAQs (Rarely Asked Questions) for the Curious

How Carbohydrates and Glycogen are Connected?

Carbohydrates and glycogen are intrinsically linked in human metabolism.

Carbohydrates, found in foods like grains, fruits, and vegetables, are broken down into glucose—a primary energy source.

Excess glucose is converted into glycogen, a polysaccharide, for storage in the liver and skeletal muscles.

This stored glycogen can be rapidly mobilized to maintain blood glucose levels during fasting or provide energy during physical activity.

Thus, glycogen serves as the body’s method of storing carbohydrates for future energy needs.

How are glycogen and glycerol are different? They sound similar and so are confusing

Glycogen and glycerol, despite their similar-sounding names, are distinct substances with different structures and functions in the body.

Glycogen:

Structure: A highly branched polysaccharide composed of numerous glucose molecules linked together.
Function: Serves as the primary storage form of glucose in animals, fungi, and bacteria. It is stored mainly in the liver and muscle tissues and can be rapidly mobilized to meet energy demands.

en.wikipedia.org

Glycerol:

Structure: A three-carbon molecule that is chemically similar to sugar; it can be thought of as half of a glucose molecule.

learn.genetics.utah.edu
Function: Acts as the backbone for triglycerides (fats) by binding with three fatty acids. During fat metabolism, triglycerides are broken down into fatty acids and glycerol, which can then be used for energy production or gluconeogenesis (the synthesis of glucose from non-carbohydrate sources).

en.wikipedia.org

In summary, while glycogen is a large, branched polymer of glucose serving as a storage form of energy, glycerol is a simple three-carbon molecule that forms the backbone of triglycerides and plays a role in lipid metabolism. Their structural differences underpin their distinct functions within the body’s energy management systems.

In summary:

Glycogen: Think of “glycogen” as the body’s “glucose generator.” It is a large, branched molecule made up of glucose units and serves as the primary storage form of glucose in the body, mainly found in the liver and muscles.
Glycerol: Remember “glycerol” as the “lipid link.” It is a simple three-carbon molecule that forms the backbone of triglycerides, the main constituents of body fat in humans and other animals.

Your Body is Mostly Just a Big Battery

Sat, 15 Mar 2025 20:41:21 +0530

I was recently reading Biohacking Lite by Andrej Karpathy, particularly the section called The Four Batteries of Your Body. For the first time in my life, health-related topics are starting to make sense to me because Karpathy explains them from first principles.

TLDR

This is my TLDR after an hour of studying the topic. Any mistakes or misunderstandings are my own, and I may update this as my knowledge improves.

Most of the human body is just battery storage that runs itself.
There are different types of batteries to power various activities:
- A quick, short-term battery that wakes up fast but finishes quickly.
- An on-demand, medium-term battery that wakes up a bit slower but works longer.
- A serious-effort, long-term battery that wakes up even slower but works much, much longer.
- A danger zone where your vital organs become the battery (!).
At the end of the day, most of your body is just a big battery.
All the batteries are used to convert ADP to ATP, which in turn powers all bodily activities.
Short-term and medium-term batteries activate quickly but are mostly anaerobic (running without oxygen).
The long-term battery is mostly aerobic (operating with oxygen).

The Detailed Version

The human body is like an iPhone with a battery pack. The only difference is that this battery pack can charge beyond 100% with an almost infinite capacity to keep adding to it.

The modern environment is filled with all sorts of foods and drinks around us, creating an oversupply of food. So, the charging outlet is always there, paired with this “always hungry” battery pack.

The technical terms for the important battery packs are: Adipose tissue & Triglycerides (fat).

These are stockpiled and sometimes synthesized, leading to increased volume. Stockpiling made sense in historic times when there were no guarantees about when one could get the next meal. However, with an abundance of food available today, this storage capacity is working against us.

From ADP to ATP

While there are many ways to stock up on adipose tissue and triglycerides, there’s only one way they get consumed: to synthesize ATP from ADP. ATP stands for adenosine triphosphate, and ADP stands for adenosine diphosphate.

ATP is very important for the body because it is the universal currency through which internal work gets done:

Transporting molecules across cell membranes.
Untying DNA against hydrogen bonds.
Moving myosin to operate muscles.
Assisting with protein synthesis.

At any point in time, there’s only a very small amount of ATP available. ADP is attached to a phosphate group to become ATP. When the phosphate group is detached, energy is released to perform any work.

So, that’s the universal work done by the body to accomplish higher-level goals. To convert ADP to ATP, there are four types of batteries available:

Super short-term battery
Short-term battery
Long-term battery
Lean body mass

Super Short-Term Battery: The Phosphocreatine System

This system provides an immediate but very limited energy buffer, allowing ATP to be rapidly regenerated. Many athletes take creatine supplements to enhance this buffer.

Stores phosphate groups attached to creatine.
Enables quick, localized recycling of ADP into ATP.
Has an extremely small capacity, so it’s not a major factor in overall energy storage.

Short-Term Battery: Glycogen {24 Hours}

Glycogen is the body’s primary short-term energy store, located in the liver and muscles. It provides about 2,000 kcal, roughly one day’s worth of energy at basal metabolic rate (BMR). However, it is inefficient due to its low energy density and high water retention.

Storage Locations:
- Liver: ~120g
- Skeletal muscle: ~400g
- Blood glucose: ~4g
Energy Yield:
- 4 kcal per gram of glycogen.
- Total glycogen energy: ~2,000 kcal.
Efficiency Issues:
- Glycogen binds ~3g of water per gram, making it a poor long-term storage medium.

Long-Term Battery: Adipose Tissue (Fat) {50-80 Days}

Fat is the body’s main high-capacity energy reserve, storing significantly more energy than glycogen. It is more than twice as energy-dense as carbohydrates and can sustain the body for weeks or even months if needed.

Energy Density:
- Fat: 9 kcal per gram (vs. glycogen’s 4 kcal/g).
Example Calculation (from my 2019 data):
- Body weight: 200 lbs
- Fat mass: 40 lbs = 18,000g
- Total energy in fat: 18,000g × 9 kcal/g = 162,000 kcal
- Survival estimate: 162,000 kcal ÷ 2,000 kcal/day = ~81 days
Comparisons for Scale:
- Equivalent to 678 sticks of dynamite (1 MJ ≈ 239 kcal).
- Nearly enough energy to fully charge a 100 kWh Tesla battery twice!

Last Resort Battery: Lean Body Mass

When fat reserves are depleted, the body breaks down muscle tissue for energy—a last-ditch survival mechanism.

Occurs only in extreme fasting or starvation.
Uses muscle proteins as the primary fuel source, converting them into glucose or ketones.
This process weakens the body over time, making it highly undesirable.

How Your Body Uses Energy

Your body constantly charges and discharges all four energy stores at different rates, depending on your activity and food intake.

After Eating (e.g., a Cookie):
- The cookie is broken down into glucose, which enters the bloodstream.
- If there is excess glucose (as there usually is with cookies), your body:
  - Stores it as glycogen in the liver and muscles.
  - Rarely, if glucose is in extreme excess, converts it to fat.
During Exercise (e.g., Jogging):
- First 3 seconds: The phosphocreatine system (1) provides immediate energy.
- Next 8-10 seconds: Glycogen (2) is used anaerobically (without oxygen).
- Longer duration:
  - Glycogen (2) and fat (3) become the main energy sources.
  - These rely on aerobic metabolism, which takes longer to activate but provides sustained energy.
  - Your body increases heart rate, breathing, and oxygen transport to keep up.
- In Starvation or Carb Deprivation: The body eventually resorts to breaking down muscle (4) for energy, a last-resort survival mechanism.

The Computer Memory Analogy (Brilliant Stuff)

This energy hierarchy can be compared to a computer’s memory hierarchy:

Phosphocreatine System (1) → L1/L2 Cache
- The fastest and most immediate energy source, but with very limited capacity.
Anaerobic Glycolysis (2) → RAM
- Offers quick access, but has limited storage and is less efficient.
Aerobic Metabolism (3) → Disk Storage
- Provides high capacity and efficiency, but is slower to access due to the need for oxygen and the transport of fatty acids from adipose tissue.

Just like in computing, moving energy around is costly, so the body prioritizes faster but smaller energy stores before tapping into slower but larger reserves.

Two Word Decisions

Fri, 14 Mar 2025 20:41:21 +0530

I’ve always been interested in decisions—two ways, really.

I wanted to make better decisions.

I wanted to make decisions in a better manner.

When it comes to making decisions better, what I care about is conserving energy while still getting a decision done and executed.

In my previous post, I explained how vagueness is the enemy of action.

In this post, I’ll share a trick to fight against vagueness and fight for clarity.

Your Willpower Is Limited

We know that willpower is a scarce resource—like a battery that gets drained throughout the day as we make decisions and take actions.

Every decision we make taxes this willpower battery.

And sometimes, we get stuck—overthinking, worrying, or somehow unable to proceed smoothly.

This usually happens when we’re faced with some sort of problem, big or small.

Especially when stuck on something, inaction can be fatal.

As noted earlier, learning is movement.

My general experience has been that I have high self-control and willpower in the mornings, when the battery is full.

But as the day progresses, my powers wane, and evenings and nights become more vulnerable.

I’m more likely to waste time, break good eating habits, slouch in my posture, or falter in other ways due to this lack of willpower.

How to Get Unstuck—Without Taxing Your Willpower

Not only do we want to get unstuck and move on to the next action—which will give us more clues to solve the problem at hand—but we also want to do it without draining our willpower reserves.

When we’re stuck, what we need is a nudge toward something slightly better.

Say I’m considering grabbing some unhealthy food, breaking my intermittent fast, or something like that.

If I’m fortunate enough to notice the moment, I can try to make my next choice a conscious two-word decision.

Health, or Taste?

Just two words. I’m asking myself: Will I let myself be guided by my desire for health or my desire for taste?

Or take another example—I could consume one more social media post or YouTube video, or I could organize my thoughts and put them into writing. I can ask myself:

Consume, or Create?

It’s Perfectly Fine to Choose the “Worse” Option Consciously

The point of the two-word decision is conscious decision-making. When we’re about to make a potentially harmful choice, it’s like prompting ourselves with a question—and if the answer is yes, that’s fine, because it’s a deliberate choice.

Here are some decisions I’ve made:

Rest, or Exercise?
Entertainment, or Business?
Sleep, or Walk?
Slump, or Sit Straight?
Worry, or Act?
Twitter, or Book?
Assume, or Calculate?
Trust, or Verify?
Rest Now, or Solve the Problem?

A Few Technicalities

These are things I personally find helpful:

Try to notice just as you’re about to do something unproductive.
Put the potentially harmful option first (Option 1), then think of a slightly better or more inspiring alternative (Option 2).
There’s no right or wrong decision here—as long as it’s conscious.

The Bottom Line

Two-word decisions can nudge you toward better choices in a better way. Give them a try and let me know if they help you.

Vagueness is the Enemy of Action

Thu, 13 Mar 2025 18:11:28 +0000

I wasted some time today—I realized it when I looked back at how the time slipped away.

Naturally, I wondered: what went wrong here?

My mind churned for a bit, wrestling with the question, until an answer popped up: vague thoughts.

See, I’d been chasing correct judgment, aiming for correct thoughts.

In that pursuit of getting it right, I ended up making things more complex.

And that complexity? It spiraled so far that I deluded myself—straying from the simple approach I could’ve taken.

After mulling it over, a kind of Grid of Action sprang to mind, and here’s how it looks:

Vague and Wrong → No action
Vague and Correct → No action
Simple and Wrong → Action
Simple and Correct → Action

Now, someone might object: “If it’s vague, how can it be judged as correct or wrong?”

Fair point—it’s tricky to pin down something so hazy.

But that’s the thing: both possibilities exist in that murkiness.

Whether it leans right or wrong in effect, it doesn’t matter—it’s a useless exercise either way. Why? Because no action comes of it.

So, here’s the takeaway: if we want to do, to stay engaged, to move, to experience life, we’ve got to keep our thoughts simple at all times.

Complexity isn’t the enemy—it can be tackled—but it’s best surmounted through a thousand simple steps.

Each one needs to be manageable, something we can actually grip, if we’re going to act at all.

So complexity is not the enemy, but vagueness definitely is - that of action.

About HippoRAG

Wed, 12 Mar 2025 20:41:21 +0530

These are some LLM-assisted exploration notes from the paper HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models

The Multi-Hop Problem in RAGs

The idea of “hops” are important in RAG.

Consider this example.

Question 1:

“Who wrote Hamlet?”

→ The answer (Shakespeare) is in one document.*

Question 2:

“Which university did the president of OpenAI attend?”

Step 1: Retrieve information on who the president of OpenAI is (e.g., Greg Brockman).
Step 2: Retrieve information on which university Greg Brockman attended (e.g., MIT).

How HippoRAG Achieves a Single-Step Multi-Hop Retrieval

Traditional RAG solutions, such as IRCoT (Iterative Retrieval Chain-of-Thought) depend on iterative retrieval - kind of like looking up docs in a loop.

With HippoRAG, two mechanisms are combined to compress these multiple hops into one:

Building a knowledge graph (KG) where concepts and relationships are indexed.
Using Personalized PageRank (PPR) to retrieve relevant paths across multiple documents in one query.

The benefits of the above combination makes HippoRAG:

Faster (avoids iterative retrieval)
More accurate (find connections that isolated retrieval steps miss)
Cheaper (reduce API calls and computation)

How HippoRAG Builds Its Knowledge Graph (KG)

HippoRAG constructs a schemaless knowledge graph from a text corpus by leveraging large language models (LLMs) for Open Information Extraction (OpenIE) and retrieval encoders for linking entities. This process enables multi-hop reasoning in a single retrieval step.

1. Offline Indexing (Building the Graph)

This step is analogous to how the human neocortex encodes memory.

✅ Extract Knowledge Graph Triples

Uses an instruction-tuned LLM (e.g., GPT-3.5) to extract subject-predicate-object triples from text.
- Example:
  Input Passage: “Steve Jobs co-founded Apple in 1976.”
  Extracted Triples:
  - (Steve Jobs, co-founded, Apple)
  - (Apple, founded_in, 1976)

✅ Create Graph Nodes & Edges

Nodes = extracted entities (noun phrases) (e.g., Steve Jobs, Apple).
Edges = relationships between entities (e.g., co-founded).

✅ Synonymy Linking (Parahippocampal Processing)

Uses retrieval encoders (e.g., Contriever, ColBERTv2) to identify similar entities (e.g., “USA” = “United States”).
Creates extra edges to connect synonyms, improving retrieval robustness.

✅ Store the Graph

The final knowledge graph consists of:
- Nodes (Entities)
- Edges (Relations & Synonyms)
- Passage Mapping (Each node is linked to the original text passage for retrieval)

2. Online Retrieval (Querying the Graph)

This step mimics the hippocampus retrieving memories.

✅ Extract Query Named Entities

The LLM identifies key entities in the query.
Example: “Which Stanford professor works on Alzheimer’s?”
- Query Entities: {Stanford, Alzheimer’s}

✅ Find Related Nodes in the Knowledge Graph

Uses retrieval encoders to find graph nodes most similar to the query entities.
Example: The query {Stanford, Alzheimer’s} matches the node {Thomas Südhof} in the KG.

✅ Personalized PageRank (PPR) for Multi-Hop Retrieval

Runs Personalized PageRank (PPR) on the graph using query nodes as starting points.
Spreads probability over connected nodes, enabling multi-hop reasoning.
Example:
- {Stanford} → {Thomas Südhof}
- {Alzheimer’s} → {Thomas Südhof}
- Final Retrieval: Thomas Südhof is a Stanford professor working on Alzheimer’s.

✅ Retrieve & Rank Passages

The most relevant passages are selected based on PPR scores.

How HippoRAG Uses PageRank to Order Results

Convert Text to a Graph

Extract entities (nodes) and relationships (edges).
Example:

1
2


(Stanford, employs, Thomas Südhof)
(Thomas Südhof, researches, Alzheimer’s)

Find Relevant Nodes
- If the query is: “Which Stanford professor studies Alzheimer’s?”
- The query matches {Stanford, Alzheimer’s} in the graph.
Run Personalized PageRank (PPR)
- Give high starting scores to query nodes (Stanford and Alzheimer’s).
- Spread scores to connected nodes (e.g., Thomas Südhof gets a high score).
Rank Passages by PageRank Score
- Passages mentioning Thomas Südhof get top rank.
- Less relevant passages rank lower.

Why This Works

Finds indirect connections (multi-hop retrieval).
Ranks based on real-world relevance rather than keyword matching.
Fast, since it’s done in one step.

RAG: Not a Silver Bullet?

Tue, 11 Mar 2025 20:01:33 +0000

A Few Ideas About RAG

As I keep pouring in efforts to build LiveAPI, I get new ideas on how to make it better from time to time.

There is a constant feeling that maybe we should have some sort of Chatbot on top of the structured API information we generate at scale.

People say:

“Nobody reads documentation these days”

“I don’t think anyone will read docs anymore”

“Nobody cares about documentation. Code is documentation”

And so on.

What they also sort of imply is that - we just want to ask a Chatbot or be surfaced with the relevant information automagically in the right place at the right time.

We may build a Chatbot - at some point in the future

But for now - this post will be about one of the approaches to building Chatbots that seems to be making noise across places like Hacker News - called “RAG”

RAG stands for Retrieval Augmented Generation

The LLM in its primordial state is like our long-term memory or impressions. It sort of has a vague recollection of facts, ideas and sentiments. But it has to be grounded with something more concrete - before a specific answer can be given.

RAG is about - helping the LLM remember better, while also helping it transform current or specific information into the framework of its answers.

RAG Flaws - Still too literal

I’ve heard from multiple quarters - with those having comparatively deeper knowledge on the topic that RAGs often underperform.

“Just tag your docs with keywords - don’t bother with RAG”

I’ve heard the above opinion expressed multiple times.

Consider the following case mentioned on an HN Thread.

There’s a textual records of various activities performed day to day by a bunch of people.

Now we are curious - “Guess the occupation of person X”

The augmentation happens purely based on the keywords present in this question.

Imagine now person X has been going to various chemistry conferences, etc but her occupation is not mentioned anywhere.

Pure RAG approaches most probably will fail here because they cannot connect “attends chemistry conference” to “occupation” - because the keywords are sort of different

What is the way around?

For things like above - we fall back to the old search and information retrieval mechanisms.

We build a semantic graph - putting related nodes together.

We do PageRank, or similar alternatives.

We essentially do search first.

The LLM layer is mostly good for “summarizing” the findings.

So - the role of something like RAG becomes much more limited, or sometimes even irrelevant.

Many AI apps may end up with Search followed by Summarization.

Rosenblatt's Perceptron (WIP)

Sun, 09 Mar 2025 23:41:21 +0530

In Smithsonian National Museum of American History - the following device is visible:

(src)

The device is called the Mark I Perceptron.

It was built in 1957.

The chief person behind construction of this device was Frank Rosenblatt

The device had 3 parts:

Stimuli receptor monitor (S)
Association machinery (A)
Response mechanism (S)

This is almost a mechanical equivalent of what Minsky sort of talks about in his Society of Mind:

This diagram depicts our sensory machinery as sending information to the brain, wherein it is projected on some inner mental movie screen. Then, inside that ghostly theater, a lurking Self observes the scene and then considers what to do. Finally, that Self may act — somehow reversing all those steps — to influence the real world by sending various signals back through yet another family of remote-control accessories. – Marvin Minsky, The Society of Mind

And this is Rosenbaltt tweaking the perceptron:

You can see the NYT reporting that the navy claimed the device would eventually:

the embryo of an electronic computer today that it expects will be able to walk, talk, see, write, reproduce itself and be conscious of its existence.

All these things aside, we will try to cull some ideas from Rosenblatt’s original paper - accessible at The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain

Memory is the foundation for higher-level capabilities (but What is it?!)

The 3 Questions (And R’s Area of Focus)

How is Stimulus represented in Storage?

R considers options on how storage may work - based on existing scholarship. The interesting scholarly decision happens right here - he picks the right architecture, the right bet so to speak.

The Critical Insight: R Makes a Bet On the Nature of Memory

Position A: Coded Representational Memory (Essentially - expecting an isolated storage of memory)

The first suggests that storage of sensory information is in the form of coded representations or images, with some sort of one-to-one mapping between the sensory stimulus and the stored pattern. According to this hypothesis, if one understood the code or ‘wiring diagram’ of the nervous system, one should, in principle, be able to discover exactly what an organism remembers by reconstructing the original sensory patterns from the ‘memory traces’ which they have left, much as we might develop a photographic negative, or translate the pattern of electrical charges in the ‘memory’ of a digital computer.

Position B: Connectionist Memory

The alternative approach, which stems from the tradition of British empiricism, hazards the guess that the images of stimuli may never really be recorded at all, and that the central nervous system simply acts as an intricate switching network, where retention takes the form of new connections, or pathways, between centers of activity. In many of the more recent developments of this position (Hebb’s ‘cell assembly,’ and Hull’s ‘cortical anticipatory goal response,’ for example) the ‘responses’ which are associated to stimuli may be entirely contained within the CNS itself. In this case, the response represents an ‘idea’ rather than an action.

Essentially - the difference in position is about direct recording vs indirect “impressions”.

Rosenblatt’s Rationale for Selecting a Connectionist Model (over Coded Representation)

Biology: Coded representation is precise, definite, and rigid – almost machine-like. Rosenblatt appreciated that different organisms remember things differently - learning from the same source! These individual variations in storage encouraged him to reject coded representation
Simplicity: Of the options available, the connectionist model is simpler and economical. So he favored such a model.
Probabilistic Pattern Matching: Rosenblatt noticed that humans and other creatures can match and work with images in diferent configurations, lighting, etc. That means - we are able to generalize the detection of objects upto an extent, classify them, and so on. He expected a more flexible model, rather than a rigid model to be able to perform this sort of work.
Recall is natural and direct in connectionist model: In the coded method - learning or retrieving information requires a special lookup process. In connectionist model, learning is merely response pathway, given a stimulus. A simpler model to get the same function.
Stronger Mathematical Basis: R embraced probabilistic models over deterministic ones from earlier (such as McCulloh-Pitts). He demonstrates how a random configuration of connections is able to achieve recognition (or learning).
Less Idealized, More biological model: He critiques earlier models - from McCulloh and Minsky - calling them too structured or algorithmic, whereas biological learning is stochastic. Connectionism better aligns with real-world learning.

In short - R betted on adaptive learning over rigid storage.

R shows that learning and memory are not different things - the connectionist model as a simpler model - collapses storage and learning into one!

Mastery's Price: Paid in Life Itself

Fri, 07 Mar 2025 18:53:27 +0000

Mastery Demands Payment in Life Units

Let’s think a bit about the 10,000 hour rule.

It says that mastery in a serious discipline demands a minimum focused investment of around 10,000 hours in deliberate practice.

Deliberate practice means - finding benchmark behaviors, breaking them down, and performing them consistently to get nearer to perfection.

10,000 hours roughly breaks down like this:

9 years of effort, given 3 hours per day, 365 days a year
13 years of effort, given 2 hours per day, 365 days a year
27 years of effort, given 1 hour per day, 365 days year

These are staggering numbers - when we look at how short our lives are.

Half our lives - we are either sleeping or working on maintenance tasks.

The other half - we’re trapped in various maladies and difficulties

and rarely give sufficient time and energy to deliberately get better at something.

For most humans - mastering even 1 serious discipline is a lifetime achievement.

The vast majority of humans reach their graves having mastered not a single serious discipline in their lives.

That means, the vast majority of humans have never experienced peak performance within themselves, they have never known what it means to fulfill even a portion of their real potentials.

Why should we care about “Mastery”?

It may sound like a vague word invented by people with too much time sometimes on their hands.

But really - it isn’t.

There are two sides to “Mastery”.

First - the less important side: The social side.

A masterful performance is usually judged in a social context.

In any era - we have the top scientists, sportsmen, politicians, actors, and so on.

We have the historical greats, Newton, Einstein, Beethoven, and on and on.

We all know their names.

They were much, much better than their contemporaries. Some of them were better than anyone ever in their fields of expertise.

It is sort of a social peak - and masters are at the top - in their areas of expertise.

But as I said - there are two sides to “Mastery”, and the social side - of being at the top of list is of lesser importance.

The More Important Side of “Mastery”

The more important side of “Mastery” is that - it hits at the core of what it means to be a human.

You may look at the animal kingdom - you’ll find no adult animal practicing getting better in any area of life, and trying to master it.

There is no such effort.

In humans - we can see a higher state and try for it. We understand this term called “potential”.

You know that if you keep practicing, and trying, you will eventually get better - and the experience of life will change for you.

That is the key driver for Mastery.

And most importantly? The experience of mastery cannot be bought or sold in the market.

Once you’ve mastered something - the experience you get out of it - is solely your own. When performing - what you go through psychologically cannot be counted through numbers.

Nobody can buy your experience. Sure, they may hire you, they may associate with you - but nobody can buy the experience of mastery.

One struggles for a decade or more - and pay a hefty price in time, effort and emotions – all for a simple experience of mastery.

And no rich person, or a powerful person, can buy or grab it for themselves.

The only currency through which mastery can be obtained is life itself.

What Mastery Stands For in Human Life

The desire and effort for mastery in any area of human life stands for intelligence, the human capacity for envisioning better states, and working for it.

It is waking up the inner human within the animal, and to help it guide towards a higher potential.

That is what Mastery means, and it is worth having a few words in our vocabularies to remind ourselves of such a great potential.

Leadership: Action That Inspires

Thu, 06 Mar 2025 18:54:11 +0000

A Definition of Leadership

These days the word leadership has become loaded with many meanings, explanations, and imagery across various media.

But in its essence - leadership simply means to lead.

The roots of the word lead emerge from an Old English word laeden which means “to go first, to guide”.

Therefore - leadership is closely tied to:

Taking action
Taking action before others (such that others follow)

There is an inherent sense of adventure - in “going first”.

But an even more important aspect of this is that - the direction and action of the leader - encourage others to follow suit.

Therefore in its elemental form: Leadership is action that inspires further action in others.

How to Identify Leadership Potential

While leadership at its core is about taking action and inspiring others to do the same - let’s take a step back.

Why does anyone take action?

In day to day life, we see two types of actions:

Asking
Proposing

Usually - you are asking because you’re not sure what needs doing.

Or maybe you don’t want to put in the work necessary to settle on a direction.

Getting more certainty often demands more action!

If you’re low on energy or tired - you’re unlikely to make demands on yourself to get clarity or conviction.

So those lacking conviction or those unwilling to act by themselves to acquire conviction are usually the ones asking.

“Shall I do X to solve Y?”

Hidden under the questions like the above - we see that there’s no conviction, only a low-effort desire to find out what needs doing.

However, the ones who lead aren’t asking these sorts of questions.

Instead - they are doing the following:

They’re experimenting and figuring things out to build personal conviction.
Once they have conviction - they are making proposals to others.
They may seek inputs, suggestions, or support from others - but they are in action, moving the agenda forward.

In short, future leaders are working to acquire conviction and then propose:

“We must do X to solve Y. I will start with doing a part of X. Do you have any suggestions on my proposal? Would you like to join me in this effort?”

Look at the amount of extra effort needed to make a proposal:

Studying the situation
Taking action to deepen understanding
Developing conviction
Making a proposal to others
Take comments, criticism, and include others in the agenda

Leadership in Practice

So what does it take to lead?

One must continuously study the field and situation.

And one must take action to validate one’s ideas and develop conviction.

The conviction must then be used to fuel a compelling proposal.

Finally, the proposal must be refined with the inputs from others…

And this will lead to a group acting together.

But it all starts with an individual’s desire and willingness to act first.

Practical Focus: Saying "Not Now" Instead of "No"

Wed, 05 Mar 2025 19:08:53 +0000

People often say, “Focus is about saying No.”

But experience tells us it is hard to say “no” to things we want to do.

And every “no” that you muster chips away at your limited supply of willpower.

So, how can we make focus practical?

That is, what does it mean to arrange for practical focus?

Replace “no” with “not now.”

How?

Many people have spoken and written about “time blocking.”

The essence of “time blocking” is not to say “no,” but rather to adapt the phrase “not now.”

Yes, you can watch a movie. But not now.
Yes, you can kill time on YouTube. But not now.
Yes, you can respond to chat/email. But not now.
Yes, you can worry about 1000 things not going the way you want. But not now.

See how simple it is?

You don’t have to outright reject a low-value activity.

You just have to delay it.

And many times, if you succeed in delaying it, you’ll also succeed in rejecting it.

In more positive terms, time blocking means the following:

A duration (4 hours of time, starting now)
The most important thing that deserves your focus
A defensive fortification mantra called “not now” directed at every distraction

A Simple Tool To Quickly Find API Entry Points Across All Your Repositories

Wed, 05 Mar 2025 23:43:58 +0530

Imagine you’re deep in a coding session, cruising through your repository, when bam—you hit an API reference that stops you cold.

You’ve got no idea where it lives, and now you’re stuck.

Sound familiar? That’s the pain I want to talk about today, and how a tool I’ve been tinkering with—LiveAPI—tackles it head-on.

This isn’t just about saving time; it’s about keeping your flow intact and your sanity in check.

Let’s break it down.

The Problem

Picture this: you’re in repository R1, poking around in file F1, line L1, and you spot a call to some API route—like /users/{id}/profile. Cool, but where does it come from? Is it in this repo? Another one? You’ve got a hunch it’s somewhere in your sprawling codebase, but pinning it down feels like searching for a needle in a haystack.

Here’s what usually happens:

You grep through the current repo. Nothing.
You vaguely recall seeing it in repository R2, maybe in file F2, around line L2—but you’re not sure.
You dig through docs (if they exist), Slack threads, or—worst case—ping a colleague who’s just as busy as you.

This isn’t just annoying; it’s a flow killer.

You’re yanked out of your groove, wasting minutes (or hours) chasing down a single endpoint.

Sometimes you figure it out, sometimes you don’t, but either way, it’s a disruption.

Multiply that by dozens of APIs across multiple repos, and you’ve got a productivity nightmare.

Pain Point	Impact
Unclear API location	Breaks focus, slows momentum
Vague recollection	Extra mental load to recall
Manual hunting	Time sink, inconsistent results
Bothering colleagues	Delays, team friction

How LiveAPI Solves the Problem

Enter LiveAPI, a tool built to cut through this mess.

It’s not some lightweight script—it’s an infrastructure-scale solution designed to map out all your REST backend APIs across every repository you throw at it.

Whether you’ve got dozens, hundreds, or even thousands of repos, LiveAPI’s got your back.

Here’s the magic:

Broad coverage: We support the top 15 languages and 60+ frameworks. Python with Flask? Java with Spring? Node.js with Express? You name it, we handle it.
Real-time tracking: LiveAPI watches your repos for changes. If an endpoint gets updated, added, or axed, it stays in sync—no stale data here.
Fast search: Need to find /users/{id}/profile? Type it in, and LiveAPI spits out the exact endpoint, parameters, and even a description if you’ve got one documented.
Source code links: One click, and you’re at the exact file and line—like jumping from R1, F1, L1 to R2, F2, L2 without breaking a sweat.

Think of it like a GPS for your APIs.

Instead of fumbling through repos or bugging your teammate, you get a clean, instant path to the source.

Here’s a quick example of what you might see:

Search Query	Result	Source
`/users/{id}/profile`	GET endpoint, returns user profile	`R2/F2#L2`
`POST /auth/login`	Authenticates user, returns token	`R3/F5#L12`

This is one of the visuals I’ve got—it’s a crisp look at how LiveAPI ties an endpoint to its origin in one glance.

Roadmap to Improve Developer Experience

LiveAPI’s already pretty handy, but we’re not stopping there.

Here’s what’s cooking to make it even better:

Intelligent chat: Search is great, but what if you could talk to your API map? We’re exploring a chat interface where you can ask things like, “Which endpoints handle user auth?” and get a conversational breakdown. It’s about building on your infrastructure, not just finding it.
IDE extensions: Imagine right-clicking an API call in your editor and pulling up its details without leaving your context. We’re working on plugins for VS Code, IntelliJ, and others to make that a reality.
Accuracy & reliability: We’re testing and tweaking every day. The goal? Make sure LiveAPI catches every endpoint, every change, every time—no misses, no lag.

The focus is on keeping it seamless.

You shouldn’t have to think about the tool; it should just work when you need it.

How LiveAPI Solves the Entry Point Problem (Reiterate)

Let’s circle back to that core headache: finding where your APIs live.

LiveAPI doesn’t just slap a band-aid on it—it rewrites the game.

By indexing every REST endpoint across all your repos, it turns a scattered hunt into a single, confident click.

Say you’re staring at that /users/{id}/profile call again. With LiveAPI:

Punch it into the search.
See the endpoint, its params, and a link to R2, F2, L2.
Jump straight there—no grep, no guesswork, no Slack pings.

This second image nails it—showing the leap from query to code in action.

It’s not just about speed (though it’s fast).

It’s about staying in the zone.

No more context-switching or mental juggling.

Whether you’re debugging, onboarding, or just curious, LiveAPI hands you the keys to your API kingdom.

Wrapping Up

If you’re tired of playing API hide-and-seek across your repositories, give LiveAPI a spin.

It’s built for developers like us—people who want answers fast and flow uninterrupted.

Check out the LiveAPI site for more, or drop a comment if you’ve got ideas to make it even better.

What’s your biggest API related horror story? I’d love to hear it.

LiveAPI Rebuilt: A Scalable and Reliable API Documentation Tool

Tue, 04 Mar 2025 18:07:06 +0000

Hi there! I’m Shrijith Venkatrama, founder of Hexmos. Right now, I’m building LiveAPI, a tool designed to make generating API documentation from your code incredibly easy.

Over the past few months, we’ve been hard at work refining LiveAPI to be more robust, scalable, and user-friendly. Here’s a look at what we’ve been up to.

What We Did in November and December

Towards the end of last year, we ramped up our outreach efforts, connecting with engineering leaders, CTOs, technical freelancers, and experienced developers. Many of them were generous enough to try out LiveAPI and provide us with invaluable feedback.

We closely observed how users onboarded, interacted with, and benefited from LiveAPI. What became clear was that while the concept was strong, the product needed significant improvements before it could truly serve the needs of professional developers. With that realization, we dedicated the past three months to making fundamental enhancements.

Identifying Key Issues

Through our interactions and testing, we uncovered several major pain points:

Handling Large Codebases: Many users experienced failures when processing large repositories, making the tool impractical for sizable projects.
Challenges with Multi-Project Repos: Codebases structured as monorepos or with multiple subprojects often encountered errors.
Pricing Concerns: Users expressed a preference for API-count-based pricing rather than a repository-based model.
Lack of Framework Support: Several critical frameworks were missing from our compatibility list, limiting adoption.
Accuracy Issues: Due to size limitations, the accuracy of generated documentation was sometimes inconsistent.
Unstable LLM API Calls: Intermittent failures occurred when making LLM-based API calls, and we lacked a robust retry mechanism.
Uninspiring Documentation Design: The generated documentation needed a more polished, professional, and visually appealing design.

Re-Architecting LiveAPI: The Solution

Initially, we thought we could address these issues with incremental fixes. However, as we dug deeper, it became clear that a more fundamental overhaul was necessary. Over the past three months, we completely re-architected LiveAPI’s backend to build a more stable and scalable system.

Event-Driven Architecture: We transitioned to a fully event-driven approach, ensuring better scalability and responsiveness.
Redis-Powered Queues and Streams: We adopted Redis to efficiently manage queues, data structures, events, and streams, significantly improving performance.
Reliable LLM Interactions: We developed new mechanisms to enhance the reliability of LLM-based API calls, minimizing failures and improving consistency.
Scalability for Large Repos: We tackled repository size limitations, enabling us to generate documentation for 150-200 APIs per project with ease. This number is expected to grow further as our optimizations continue.
Future-Proof Infrastructure: The event-driven architecture allows us to scale dynamically based on demand, ensuring a seamless experience for users.
Enhanced Documentation Design: Improving the documentation layout and visual appeal required a steep learning curve for our team, but the result is a much more engaging and professional output.

The New and Improved LiveAPI: What’s Next?

With these changes in place, LiveAPI now offers:

Higher Reliability: Automated document generation is now far more stable and dependable.
Better Support for Large Repositories: Even complex, multi-project repos can be processed with ease.
Visually Appealing Documentation: The new design makes API documentation more readable and professional.
Improved Accuracy: The enhancements result in more precise and complete API documentation.
Support 60+ Frameworks: We now support the top 15 languages and around 60 frameworks within them

We’re incredibly excited about these improvements and can’t wait to share them with our users. Stay tuned for the official release of the new LiveAPI, and if you haven’t tried it yet, now’s a great time to check it out!

Don't Add More Garbage to the World

Mon, 03 Mar 2025 18:55:43 +0000

Steve Jobs said taste is paramount to business leaders who wish to do well.

I think that’s a bit too abstract to put into practice on a day-to-day basis.

Instead, I prefer thinking of it this way: “Let’s not put more garbage into the world.”

That is, it is a virtue to be critical about one’s own works.

When our standards are high, and we refuse to pollute the lives of others due to greed, impatience, and such vices, we automatically imbibe “taste.”

No esoteric or mystical skill required this way.

Learning is Movement

Mon, 03 Mar 2025 18:03:05 +0000

Learning is movement, radical change. It is not about sitting still and absorbing information. It is about moving forward and making things happen.

Focusing Questions: The Key to Getting Things Done

Sun, 02 Mar 2025 19:38:46 +0000

There are essentially two types of people: those who are successful and those who ask defocusing questions.

A defocusing question is something like: “What can I do about this?”. This creates a huge list of busywork for you – which keeps you running around for a long time with very little results to show for it.

Instead, try a FOCUSING question: “What is ONE thing if I do, everything else becomes easier or irrelevant?”

You have to be really careful about it. If you pick an actually important thing, your life will become so much better because of this.

A Friendly Step-by-Step Tutorial on the Vanishing Gradient Problem

Sat, 01 Mar 2025 22:46:54 +0530

Hi there! I’m Shrijith Venkatrama, founder of Hexmos. Right now, I’m building LiveAPI, a tool that makes generating API docs from your code ridiculously easy.

In this hands-on tutorial, we’re going to explore the vanishing gradient problem in deep neural networks—a tricky issue that can slow down or even stop learning in early layers.

You can run the sample code in a Jupyter Notebook, see diagrams, and have those “aha!” moments as we go.

We’ll use Python with NumPy (and a touch of PyTorch later) to keep things clear and practical.

By the end, you’ll also see how the ReLU activation function can save the day!

Let’s dive in, step by step, with bite-sized code snippets and plenty of explanations then.

Setting Up Your Environment

First, let’s get our tools ready. We’ll use NumPy for calculations and Matplotlib for visualizations. Open a Jupyter Notebook and run this:

1
2


import numpy as np
import matplotlib.pyplot as plt

Now, let’s define the sigmoid activation function and its derivative, since they’re central to understanding the vanishing gradient problem.

1
2
3
4
5


def sigmoid(x):
 return 1 / (1 + np.exp(-x))

def sigmoid_derivative(x):
 return sigmoid(x) * (1 - sigmoid(x))

What’s Happening?

The sigmoid function squashes any input into a range between 0 and 1.
Its derivative (which we’ll use in backpropagation) is always small—its maximum value is 0.25. This tiny derivative is a big clue to why gradients vanish!

Let’s visualize sigmoid and its derivative to get a feel for them:

1
2
3
4
5
6
7
8
9


x = np.linspace(-5, 5, 100)
plt.plot(x, sigmoid(x), label='Sigmoid')
plt.plot(x, sigmoid_derivative(x), label='Derivative')
plt.legend()
plt.title('Sigmoid and Its Derivative')
plt.xlabel('Input')
plt.ylabel('Output')
plt.grid(True)
plt.show()

Aha Moment:

Notice how the derivative peaks at 0.25 and drops to near 0 for large positive or negative inputs. When we multiply these small values across layers, gradients can shrink fast.

Building a Tiny Neural Network

Let’s create a simple network with:

1 input neuron (value = 0.5)
2 hidden layers (1 neuron each)
1 output neuron (target = 0.8)

We’ll set all weights to 0.5 and biases to 0 for simplicity.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


# Input and target
input_data = np.array([[0.5]])
target_output = np.array([[0.8]])

# Weights and biases
weights1 = np.array([[0.5]]) # Input to Hidden Layer 1
bias1 = np.array([[0]])
weights2 = np.array([[0.5]]) # Hidden Layer 1 to Hidden Layer 2
bias2 = np.array([[0]])
weights3 = np.array([[0.5]]) # Hidden Layer 2 to Output
bias3 = np.array([[0]])

Each arrow has a weight of 0.5, and each neuron uses the sigmoid function

Forward Pass—Making a Prediction

Let’s compute the output step by step. Run each line and see how the signal flows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


# Hidden Layer 1
layer1_input = np.dot(weights1, input_data) + bias1 # 0.5 * 0.5 + 0 = 0.25
layer1_output = sigmoid(layer1_input) # ≈ 0.562

# Hidden Layer 2
layer2_input = np.dot(weights2, layer1_output) + bias2 # 0.5 * 0.562 ≈ 0.281
layer2_output = sigmoid(layer2_input) # ≈ 0.570

# Output Layer
output_input = np.dot(weights3, layer2_output) + bias3 # 0.5 * 0.570 ≈ 0.285
output = sigmoid(output_input) # ≈ 0.571

print(f"Predicted Output: {output[0][0]:.3f}, Target: {target_output[0][0]}")

Aha Moment:
Our prediction (≈ 0.571) is way off the target (0.8). We need to adjust the weights, but that depends on gradients. Let’s see if they’re strong enough to help!

Compute the Loss

Let’s measure how bad our prediction is using mean squared error:

1
2


loss = 0.5 * np.power(output - target_output, 2)
print(f"Loss: {loss[0][0]:.3f}")

Plot the loss:

1
2
3
4
5


plt.figure(figsize=(4, 3))
plt.plot([0], [loss], 'ro')
plt.title('Loss')
plt.ylabel('Loss Value')
plt.show()

What’s Happening?

The loss (≈ 0.026) shows our error. Backpropagation will use gradients to reduce this, but let’s see if those gradients hold up.

Backward Pass - Where Gradients Vanish

Now, we’ll calculate gradients starting from the output and moving backward. This is where the vanishing gradient problem shows up!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


# Output Layer Gradients
d_loss_d_output = output - target_output # ≈ -0.229
d_loss_d_output_input = d_loss_d_output * sigmoid_derivative(output_input) # ≈ -0.056
gradient_weights3 = d_loss_d_output_input * layer2_output # ≈ -0.032

# Hidden Layer 2 Gradients
error_layer2 = np.dot(weights3.T, d_loss_d_output_input) # ≈ -0.028
d_loss_d_layer2_input = error_layer2 * sigmoid_derivative(layer2_input) # ≈ -0.007
gradient_weights2 = d_loss_d_layer2_input * layer1_output # ≈ -0.004

# Hideen Layer 1 Gradients

error_layer1 = np.dot(weights2.T, d_loss_d_layer2_input) # ≈ -0.0035
d_loss_d_layer1_input = error_layer1 * sigmoid_derivative(layer1_input) # ≈ -0.00085
gradient_weights1 = d_loss_d_layer1_input * input_data # ≈ -0.000425

Aha Moment:
Look at the gradient sizes:

Output layer: ≈ -0.032
Hidden Layer 2: ≈ -0.004
Hidden Layer 1: ≈ -0.000425

They’re shrinking fast! Let’s plot them:

1
2
3
4
5
6


gradients = [abs(gradient_weights1[0][0]), abs(gradient_weights2[0][0]), abs(gradient_weights3[0][0])]
layers = ['Layer 1', 'Layer 2', 'Layer 3']
plt.bar(layers, gradients)
plt.title('Gradient Magnitudes with Sigmoid')
plt.ylabel('Magnitude')
plt.show()

Insight:
The deeper we go (toward Layer 1), the tinier the gradients get. This is the vanishing gradient problem—early layers barely learn because their updates are so small!

Why Do Gradients Vanish?

Here’s the key: each gradient is multiplied by the sigmoid derivative (max 0.25). Across layers, it’s like:

Layer 3: gradient
Layer 2: gradient × <0.25
Layer 1: gradient × <0.25 × <0.25

Intuition:
Imagine passing a message through a chain of people, each whispering quieter. By the time it reaches the start, it’s almost silent. That’s what’s happening to our gradients!

How ReLU Solves the Vanishing Gradient Problem

Let’s switch to ReLU (Rectified Linear Unit), defined as:

$$ReLU(x) = max⁡(0,x)$$

with a derivative of 1 for $x>0$. This doesn’t shrink gradients! Let’s define it:

1
2
3
4
5


def relu(x):
 return np.maximum(0, x)

def relu_derivative(x):
 return np.where(x > 0, 1, 0)

ReLU Forward Pass

Redo the forward pass with ReLU:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


layer1_input = np.dot(weights1, input_data) + bias1 # 0.25
layer1_output = relu(layer1_input) # 0.25 (since 0.25 > 0)

layer2_input = np.dot(weights2, layer1_output) + bias2 # 0.5 * 0.25 = 0.125
layer2_output = relu(layer2_input) # 0.125

output_input = np.dot(weights3, layer2_output) + bias3 # 0.5 * 0.125 = 0.0625
output = relu(output_input) # 0.0625

print(f"ReLU Output: {output[0][0]:.4f}")

Note:
For simplicity, we used ReLU everywhere. In practice, the output layer might be linear for regression, but this shows the idea.

ReLU Backward Pass

Now, compute gradients:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


d_loss_d_output = output - target_output # 0.0625 - 0.8 ≈ -0.7375
d_loss_d_output_input = d_loss_d_output * relu_derivative(output_input) # -0.7375 * 1 = -0.7375
gradient_weights3 = d_loss_d_output_input * layer2_output # ≈ -0.0922

error_layer2 = np.dot(weights3.T, d_loss_d_output_input) # ≈ -0.36875
d_loss_d_layer2_input = error_layer2 * relu_derivative(layer2_input) # ≈ -0.36875
gradient_weights2 = d_loss_d_layer2_input * layer1_output # ≈ -0.0922

error_layer1 = np.dot(weights2.T, d_loss_d_layer2_input) # ≈ -0.184375
d_loss_d_layer1_input = error_layer1 * relu_derivative(layer1_input) # ≈ -0.184375
gradient_weights1 = d_loss_d_layer1_input * input_data # ≈ -0.0922

Aha Moment:
Check the gradients:

Layer 3: ≈ -0.0922
Layer 2: ≈ -0.0922
Layer 1: ≈ -0.0922

They’re all the same size! Plot them:

1
2
3
4
5


gradients_relu = [abs(gradient_weights1[0][0]), abs(gradient_weights2[0][0]), abs(gradient_weights3[0][0])]
plt.bar(layers, gradients_relu)
plt.title('Gradient Magnitudes with ReLU')
plt.ylabel('Magnitude')
plt.show()

Insight:
ReLU’s derivative of 1 (for positive inputs) keeps gradients strong, so early layers can learn just as well as later ones. No vanishing here!

Bonus: Trying it With PyTorch

Let’s quickly see this with PyTorch for a modern twist. Install PyTorch if you haven’t (pip install torch), then run:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


import torch

# Define inputs and weights as tensors with gradients
x = torch.tensor([[0.5]], requires_grad=False)
target = torch.tensor([[0.8]])
w1 = torch.tensor([[0.5]], requires_grad=True)
w2 = torch.tensor([[0.5]], requires_grad=True)
w3 = torch.tensor([[0.5]], requires_grad=True)

# Forward pass with sigmoid
h1 = torch.sigmoid(w1 @ x)
h2 = torch.sigmoid(w2 @ h1)
out = torch.sigmoid(w3 @ h2)

# Loss
loss = 0.5 * (out - target) ** 2
loss.backward()

# Print gradients
print(f"Gradient w1: {w1.grad.item():.6f}")
print(f"Gradient w2: {w2.grad.item():.6f}")
print(f"Gradient w3: {w3.grad.item():.6f}")

Output:

1
2
3


Gradient w1: -0.000424
Gradient w2: -0.003870
Gradient w3: -0.032004

You’ll see gradients shrinking from w3 to w1, just like before. Now try ReLU:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


# Reset gradients
w1.grad = None
w2.grad = None
w3.grad = None

# Forward pass with ReLU
h1 = torch.relu(w1 @ x)
h2 = torch.relu(w2 @ h1)
out = torch.relu(w3 @ h2)

# Loss and backward
loss = 0.5 * (out - target) ** 2
loss.backward()

print(f"ReLU Gradient w1: {w1.grad.item():.6f}")
print(f"ReLU Gradient w2: {w2.grad.item():.6f}")
print(f"ReLU Gradient w3: {w3.grad.item():.6f}")

Output:

1
2
3


ReLU Gradient w1: -0.092188
ReLU Gradient w2: -0.092188
ReLU Gradient w3: -0.092188

Aha Moment:
PyTorch confirms it—ReLU keeps gradients steady, while sigmoid lets them vanish.

Conclusion

You’ve just walked through the vanishing gradient problem.

With sigmoid, gradients shrink exponentially in deeper layers, slowing learning.

ReLU fixes this by keeping gradients robust.

Try tweaking the network—add layers or change weights—and watch how gradients behave.

You’ve got the tools and intuition now—happy experimenting!

Making A Simple Image With Tensors (Starting from Scratch)

Fri, 28 Feb 2025 23:11:51 +0530

Hi there! I’m Shrijith Venkatrama, founder of Hexmos. Right now, I’m building LiveAPI, a tool that makes generating API docs from your code ridiculously easy.

Tensors are a fundamental concept in machine learning and deep learning.

In this tutorial, we will explore tensors in a fun and light-hearted way to gain some familiarity

What is a Tensor?

A tensor is a multi-dimensional array that generalizes familiar concepts:

Scalar: A single number (0D tensor).
Vector: A list of numbers (1D tensor).
Matrix: A grid of numbers (2D tensor).
Higher-dimensional tensors: Structures with 3 or more dimensions (e.g., 3D, 4D).

Think of tensors as containers for organizing data in multiple dimensions, like stacking grids or cubes.

The Challenge: Draw a Red “0” on Grey Background using Tensors

To start with, we are going to set a fairly simple goal.

We will learn a bit about tensors and then try to draw a red “0” on a grey background using the new knowledge we acquire.

For our goal:

Grayscale = 2D tensor (intensity values).
Color (red “0”) = 3D tensor (RGB).

Scalars - Single Pixels

A scalar is one number, like a pixel’s intensity.

1
2


scalar = 128 # Medium gray
print("Scalar (Pixel Intensity):", scalar)

128 is medium gray (0 = black, 255 = white).
This is a 0D tensor, the simplest building block.

Vectors - Rows of Pixels

A vector (1D tensor) is a sequence of numbers, like a row of pixels.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


vector = [0, 64, 128, 192, 255]
print("Vector (Row of Pixels):", vector)

import matplotlib.pyplot as plt
plt.plot(vector, [0]*len(vector), 'ro')
plt.title("Vector: Row of Pixel Intensities")
plt.xlabel("Pixel Intensity (0 = Black, 255 = White)")
plt.ylabel("Position in Row")
plt.grid(True)
plt.show()

In this image - you can see the pixel intensity going from 0 to 255.

0 means - black, and 255 means white, and anything in between is a shade of grey.

Matrices - Grayscale Images

A matrix (2D tensor) is a grid of numbers, like a grayscale image.

1
2
3
4
5
6
7
8
9


matrix = [[0, 64], [128, 192]]
print("Matrix (Grayscale Image):", matrix)

plt.imshow(matrix, cmap='gray')
plt.title("2x2 Grayscale Image")
plt.xlabel("Column")
plt.ylabel("Row")
plt.colorbar(label="Intensity (0 = Black, 255 = White)")
plt.show()

Each cell in this matrix represents a pixel’s intensity.
The colorbar shows the intensity values.

3D Tensors - Adding Color

A 3D tensor has three 2D matrices for RGB channels.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


red_channel = [[255, 0], [0, 0]] # Red in top-left
green_channel = [[0, 0], [0, 0]]
blue_channel = [[0, 0], [0, 0]]
color_image = [red_channel, green_channel, blue_channel]

print("Color Image (RGB Channels):", color_image)

fig, axs = plt.subplots(1, 3, figsize=(10, 3))
for i, (channel, color) in enumerate(zip(color_image, ['Red', 'Green', 'Blue'])):
 axs[i].imshow(channel, cmap='gray')
 axs[i].set_title(f"{color} Channel")
 axs[i].set_xlabel("Column")
 axs[i].set_ylabel("Row")
plt.tight_layout()
plt.show()

Visualization:

Three subplots: RGB channels.
Red Channel: 255 at top-left (white); others 0 (black).
Green/Blue: All 0.

Explanation:

3x2x2 tensor where only red is active.

Combining Channels

Combine RGB channels into a color image.

1
2
3
4
5
6
7
8


import numpy as np
color_image_array = np.stack(color_image, axis=2)

plt.imshow(color_image_array)
plt.title("Combined 2x2 Color Image")
plt.xlabel("Column")
plt.ylabel("Row")
plt.show()

Visualization:

Image: Red top-left, black elsewhere.
Axes: 2x2 grid.

Explanation:

RGB = (255, 0, 0) is red; (0, 0, 0) is black

Grayscale Background

Create a 5x5 grayscale background.

1
2
3
4
5
6
7
8
9


background = [[128 for _ in range(5)] for _ in range(5)]
print("Grayscale Background:", background)

plt.imshow(background, cmap='gray')
plt.title("5x5 Grayscale Background")
plt.xlabel("Column")
plt.ylabel("Row")
plt.colorbar(label="Intensity (0 = Black, 255 = White)")
plt.show()

Visualization:

Uniform gray: All 128.
Axes: 5x5 grid.

Explanation:

A 2D tensor for a medium gray background.

Drawing a Red “0”

Now, draw a red “0” on the 5x5 background using a 3D tensor. We’ll define “0” as a small oval.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30


# Define "0" as an oval in 5x5 grid
zero_positions = [(1,1), (1,2), (1,3), (2,1), (2,3), (3,1), (3,2), (3,3)] # Oval shape

# Initialize 5x5 RGB channels
red_channel = [[0 for _ in range(5)] for _ in range(5)]
green_channel = [[0 for _ in range(5)] for _ in range(5)]
blue_channel = [[0 for _ in range(5)] for _ in range(5)]

# Set red "0" pixels
for pos in zero_positions:
 red_channel[pos[0]][pos[1]] = 255

# Set grayscale background (RGB = 128, 128, 128)
for i in range(5):
 for j in range(5):
 if (i, j) not in zero_positions:
 red_channel[i][j] = 128
 green_channel[i][j] = 128
 blue_channel[i][j] = 128

# Combine into 3D tensor
color_image = [red_channel, green_channel, blue_channel]
color_image_array = np.stack(color_image, axis=2)

# Visualize
plt.imshow(color_image_array)
plt.title("Red '0' on Grayscale Background")
plt.xlabel("Column")
plt.ylabel("Row")
plt.show()

Visualization:

Image: A red oval “0” on gray.
Red pixels: RGB = (255, 0, 0) at zero_positions.
Gray background: RGB = (128, 128, 128) elsewhere.
Axes: 5x5 grid.

Explanation:

New zero_positions: Forms a rough oval:
- Top: (1,1), (1,2), (1,3)
- Sides: (2,1), (2,3)
- Bottom: (3,1), (3,2), (3,3)
This 5x5x3 tensor places red where the “0” is and gray everywhere else.

Conclusion

So - that’s it.

We progressed from:

Scalars: Pixel intensities.
Vectors: Rows of pixels.
Matrices: Grayscale images.
3D Tensors: Color images.

At each step, we visualized the data to understand its structure and content.

Hope this gave you a good intuition about tensors!

How many Indian Companies Generate 1M+ USD in ARR?

Thu, 27 Feb 2025 16:35:55 +0530

The Problem

As a founder of a startup building from India, I often ask myself what is the situation of Indian startups in the global market.

In startup literature across the web, we often hear about the 1 million in USD ARR as a benchmark for a startup to be considered a “serious” company.

So I wanted to know, how many Indian startups already possess 1M+ USD in ARR?

A Bit of Terminological Groundwork

When we talk about startups and revenues, sometimes, the terminology can get a bit confusing.

For instance, the top line revenue of a startup is called “ARR” or Annual Recurring Revenue.

But when we talk about MSMEs, the top line revenue is called “Turnover”.

In many accounting contexts, we use numbers like “EBITDA” or “EBIT” to get a sense of the profitability of a company.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


Revenue/Sales/Turnover/ARR - Cost of Goods Sold (To Suppliers)
 = Gross Profit

Gross Profit - Salary (To Employees) = EBITDA (Earnings Before
 Interest, Taxes, Depreciation and Amortization)

EBITDA - (Equipment Depreciation + Intangible Amortization)
 = EBIT (Earnings Before Interest and Taxes)

EBIT - Interest (To Creditors) = EBT (Earnings Before Taxes)

EBT - Taxes (To Government) = EAT (Earnings After Taxes)

EAT - Dividends (To Shareholders) = Retained Earnings

The Data

We fetch some grounding numbers from IBEF and plot it with a bar chart.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44


micro = 3_93_18_355
small = 6_08_935
medium = 55_488

all = micro + small + medium
print("Total active MSMEs: ", all)
print("micro %: ", (micro * 1.0)/all)
print("small %: ", (small * 1.0)/all)
print("medium %: ", (medium * 1.0)/all)

# Import required libraries
import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd

# Ensure plots show inline in Google Colab
%matplotlib inline

# Data
categories = ["Micro", "Small", "Medium"]
percentages = [98.3, 1.5, 0.2]
counts = ["3,93,18,355", "6,08,935", "55,488"]

# Create a DataFrame
df = pd.DataFrame({"Category": categories, "Percentage": percentages, "Count": counts})

# Set Seaborn style
sns.set_style("whitegrid")

# Create the bar plot
plt.figure(figsize=(8, 5))
ax = sns.barplot(x="Category", y="Percentage", data=df, palette="viridis", edgecolor="black")

# Annotate with exact percentages and counts
for i, (percent, count) in enumerate(zip(df["Percentage"], df["Count"])):
 ax.text(i, percent + 1, f"{percent}%\n({count})", ha="center", fontsize=12, fontweight="bold")

# Labels and title
plt.xlabel("")
plt.ylabel("Percentage", fontsize=12)
plt.title("MSME Distribution in India", fontsize=14, fontweight="bold")

# Show plot inline in Colab
plt.show()

The result is very clear: the vast majority of MSMEs are micro businesses.

Classifying MSMEs

Criteria	Turnover	Investment
Micro	Rs. 5 crores (US$ 610,000)	Less than Rs. 1 crore (US$ 120,000)
Small	Rs. 50 crores (US$ 6.1 million)	More than Rs. 1 crore (US$ 120,000) but less than Rs. 10 crore (US$ 1.2 million)
Medium	Rs. 250 crores (US$ 30.4 million)	More than Rs. 10 crore (US$ 1.2 million), but less than Rs. 50 crore (US$ 6.1 million)

For our question, we are interested in only Small and Medium MSMEs.

The reason is simple: we are looking for companies that are already generating 1M+ USD in ARR. Micro businesses do not fall in this category.

Estimating the Number of MSMEs with > 8.3 Cr INR in ARR

Small MSMEs

50% above Rs. 5 crore → 304,467
50% of those above Rs. 8.3 crore → 152,234

Medium MSMEs

55,488 (all above Rs. 50 crore)

Large Companies

Estimated between 10,000 to 20,000

Monte Carlo Simulation

Uses random sampling to estimate a 95% confidence interval for large companies.

Final Estimate

Total Range: ~ 100,000 to 200,000
95% CI for Large Companies: A refined estimate with confidence bounds.

This provides a quantitative estimate while acknowledging uncertainty.

Code to Visualize the Estimates

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


# Calculate the number of MSMEs in each category
import numpy as np
import scipy.stats as stats

# Given data
small_total = 608935
small_above_5cr = small_total * 0.5 # 50% above Rs. 5 crore
small_above_8_3cr = small_above_5cr * 0.5 # 50% above Rs. 8.3 crore

medium_total = 55488 # All medium MSMEs are above Rs. 50 crore

# Large companies estimate
large_lower = 10000
large_upper = 20000

# Estimate range of total count
estimated_total_lower = small_above_8_3cr + medium_total + large_lower
estimated_total_upper = small_above_8_3cr + medium_total + large_upper

# Confidence interval for large companies (assuming uniform distribution)
large_sample = np.random.uniform(large_lower, large_upper, 10000) # Monte Carlo sampling
confidence_interval = np.percentile(large_sample, [2.5, 97.5]) # 95% confidence interval

# Print results
print(f"Estimated Small MSMEs above Rs. 8.3 crore: {int(small_above_8_3cr):,}")
print(f"Medium MSMEs: {int(medium_total):,}")
print(f"Estimated Large Companies: {int(large_lower):,} to {int(large_upper):,}")
print(f"Total Estimate: {int(estimated_total_lower):,} to {int(estimated_total_upper):,}")
print(f"95% Confidence Interval for Large Companies: {confidence_interval[0]:,.0f} to {confidence_interval[1]:,.0f}")

Graphing:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64


import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt

# Seaborn styling
sns.set_theme(style="whitegrid")

# Given data
categories = ["Small MSMEs (>8.3cr)", "Medium MSMEs (>50cr)", "Large Companies (>100cr)"]
small_total = 608935
small_above_5cr = small_total * 0.5 # 50% above Rs. 5 crore
small_above_8_3cr = small_above_5cr * 0.5 # 50% above Rs. 8.3 crore
medium_total = 55488 # All medium MSMEs are above Rs. 50 crore
large_lower, large_upper = 10000, 20000 # Large company estimate

# Monte Carlo simulation for Large Companies CI
large_sample = np.random.uniform(large_lower, large_upper, 10000)
confidence_interval = np.percentile(large_sample, [2.5, 97.5]) # 95% confidence interval

# Dataframe for plotting
data = pd.DataFrame({
 "Category": categories,
 "Count": [small_above_8_3cr, medium_total, (large_lower + large_upper) / 2], # Use mid-point for Large
 "Lower": [small_above_8_3cr, medium_total, confidence_interval[0]],
 "Upper": [small_above_8_3cr, medium_total, confidence_interval[1]]
})

# Calculate xerr (error bars)
data["xerr"] = data["Upper"] - data["Count"] # Only positive error bars

# Plot
plt.figure(figsize=(10, 5))
ax = sns.barplot(
 data=data,
 y="Category",
 x="Count",
 hue="Category", # Assign hue to avoid warning
 palette=["#1f77b4", "#ff7f0e", "#2ca02c"], # Custom colors
 errorbar=None, # Disable automatic error bars
 capsize=0.2
)

# Manually add error bars for Large Companies
plt.errorbar(
 x=data["Count"], y=range(len(data)),
 xerr=data["xerr"], fmt="none", ecolor="black", elinewidth=1.5, capsize=3
)

# Labels & Titles
plt.xlabel("Estimated Count")
plt.ylabel("Enterprise Category")
plt.title("Distribution of MSMEs & Large Companies in India", fontsize=14, fontweight="bold")
plt.legend([], [], frameon=False) # Remove extra legend

# Add exact numbers as text
for i, (value, category) in enumerate(zip(data["Count"], data["Category"])):
 plt.text(value * 1.05, i, f"{int(value):,}", va='center', fontsize=12)

plt.xlim(0, max(data["Upper"]) * 1.2) # Scale x-axis properly
plt.tight_layout()

# Show plot
plt.show()

The Result

The Conclusion

The toal number of MSMEs with 1M+ USD in ARR is somewhere around 200K organizations. That is a mere 0.5% of the 4 crore businesses active in the country.

It could be interesting to compare India’s data to:

Other developing countries of the present day
Recently developed countires of the past 50-75 years

And see what we can learn from them.