How I Solved a 7-Day Calculation Problem in a weekend Jithin Sankar Follow 3 min read · 3 days ago 3 days ago -- Listen Share
In a previous project, my team was building a SaaS application. The scenario was this: we had a slider for setting a price, from $0.00 to $10.00. When you moved the slider, the app would show you the projected sales for that price. The slider input was combined with two other attributes, Region and SKU name.
The problem was performance. Each time the slider moved, it would take a full 15 seconds to get a response from the ML model. We knew better hardware could fix it, but we had serious resource constraints due to budget issues.
The obvious solution was to pre-compute and cache(I built a pypi package for it) all the possible values. Let’s look at the numbers.
We had 10 SKUs, 4 regions, and the slider had two-decimal precision ($0.00, $0.01, …, $9.99, $10.00), which is 1,001 steps.
The total number of combinations would be:
10 SKUs x 4 regions x 1,001 price points = 40,040 requests.
If each request takes 15 seconds, the total time would be:
40,040 x 15 seconds = 600,600 seconds. That’s about 6.9 days.
Our demo was in 4 days. 6.9 days was not feasible.
So, I thought, let’s do it this way: let’s pre-compute just the odd values, like $0.01, $0.03, and so on.
The probability of the person doing the demo hitting a price with an odd number of cents is 50%. This meant 50 out of 100 times, it would read from the pre-computed cache and give results instantaneously. For the rest, we’d have to wait. During the demo, our team member could surely rig this by picking a cached value, but if the client wanted to play with the values themselves, the whole team would be in trouble. A 50% chance of failure was a risk, but we could go further.
There was one thing I had observed during all our demo trials: when people move a slider, they almost always try to be in the middle. I have never seen anyone working on the extreme ends, like a number near zero or ten.
I remembered this from my engineering days at the College of Engineering, Trivandrum. It’s called the Normal Distribution, or Gaussian distribution. It’s based on the idea that data near the mean (the average) is more frequent than data far from the mean.
A normal distribution curve
The basic idea: most events happen in the middle, fewer at the edges.
Gaussian distribution in this scenario
I figured I could apply some “Gaussian sampling” here (I made up the term, then checked on the internet and saw it exists, so I felt I was on the right path). The idea was to take more values from the middle of the range with high precision (two decimal places) and, as I moved toward the ends of the slider, take fewer values with less precision.
Let’s say I aimed to compute roughly 1/3rd of the total values, but strategically chosen. The new math looked like this:
10 SKUs x 4 regions x ~333 strategic price points x 15 seconds = ~199,800 seconds.
This is 55.5 hours, or about 2.3 days.
I started the computation script on a Friday night, and by Monday morning, it was ready.
As expected, during the demo, the demonstrator put the slider somewhere in the middle. That value was cached and showed the result instantaneously. Fortunately, the client didn’t have any unexpected options to try. All went well.
I wasn’t lucky. I trust my numbers.