mathematics | Mohammad Moshtaghi

Elegant Derivatives of Large Products

Tue, 30 Nov 2021 11:57:40 -0700

An ongoing research project on modeling computer security has thrown me back into calculus for the first time since my undergraduate differential equations course in 2014, which as an aside was the only college course that forced me to buy the professor’s custom textbook just to get access to homework problems. (I’m still mad about that.) Anyway, my research led me to take the derivative of an expression of the form:

$$\prod_{i=1}^n f_i(x)$$

For the uninitiated, $\prod$ is a mathematical symbol meaning “product” (and is analogous to the $\sum$ symbol meaning “sum”). It says to look at each $f_i$ term as $i$ goes from $1, 2, \ldots, n$ and multiply them all together:

$$\prod_{i=1}^n f_i(x) = f_1(x) \times f_2(x) \times \cdots \times f_n(x)$$

Taking this derivative is something high-school me was much better prepared for than current postdoc me, but after some head scratching I found the elegant solution:

$$\frac{d}{dx} \prod_{i=1}^n f_i(x) = \prod_{i=1}^n f_i(x) \cdot \sum_{i=1}^n \frac{f_i'(x)}{f_i(x)}$$

As is common in mathematics, there’s a direct but tedious way to get this answer and another more elegant way to get the same thing. I’m not the first to derive this solution, but I’m writing it up for posterity because I will inevitably forget this trick in a few days. A PDF of the two derivations without all the handholding I’m about to do is available here.

Method 1: Product Rule

I had to sweep away years of mental cobwebs to remember the product rule for derivatives. My expression is a product, after all, so why not start there? The product rule states:

$$\frac{d}{dx}(u \cdot v) = u' \cdot v + u \cdot v'$$

This works for the product of two terms, but $\prod_{i=1}^n f_i(x)$ has $n$ terms. Let’s try peeling off terms one-by-one, starting with $f_1(x)$, and then applying the product rule.

$$\begin{align*} \frac{d}{dx} \prod_{i=1}^n f_i(x) &= \frac{d}{dx} \left(f_1(x) \cdot \prod_{i=2}^n f_i(x)\right) \\ &= f_1'(x) \cdot \prod_{i=2}^n f_i(x) + f_1(x) \cdot \frac{d}{dx} \left(\prod_{i=2}^n f_i(x)\right) \end{align*}$$

That’s progress, but now we have to repeat the process to deal with the $\frac{d}{dx} \left(\prod_{i=2}^n f_i(x)\right)$ term, peeling off $f_2(x)$ and applying the product rule again:

$$\begin{align*} \frac{d}{dx} \prod_{i=1}^n f_i(x) &= f_1'(x) \cdot \prod_{i=2}^n f_i(x) + f_1(x) \cdot \frac{d}{dx} \left(\prod_{i=2}^n f_i(x)\right) \\ &= f_1'(x) \cdot \prod_{i=2}^n f_i(x) + f_1(x) \\ &\quad \cdot \left(f_2'(x) \cdot \prod_{i=3}^n f_i(x) + f_2(x) \cdot \frac{d}{dx} \left(\prod_{i=3}^n f_i(x)\right)\right) \end{align*}$$

This expression’s going to have more layers than an ogre! If we were to do the peel-and-product rule strategy $n-1$ times, we would arrive at:

$$\begin{align*} \frac{d}{dx} \prod_{i=1}^n f_i(x) &= f_1'(x) \cdot \prod_{i=2}^n f_i(x) + f_1(x) \\ &\quad \cdot \left(f_2'(x) \cdot \prod_{i=3}^n f_i(x) + f_2(x) \right. \\ &\quad \cdot \left(f_3'(x) \cdot \prod_{i=4}^n f_i(x) + f_3(x) \cdots + f_{n-2}(x) \right. \\ &\quad \left. \left. \cdot \left(f_{n-1}'(x) \cdot f_n(x) + f_n'(x) \cdot f_{n-1}(x)\right) \cdots \right) \right) \end{align*}$$

Do you see the pattern? If we distribute the $f_i(x)$ terms at the end of each of the lines above into the parentheses that immediately follow,

$$\begin{align*} \frac{d}{dx} \prod_{i=1}^n f_i(x) &= f_1'(x) \cdot \prod_{i=2}^n f_i(x) + f_2'(x) \cdot f_1(x) \cdot \prod_{i=3}^n f_i(x) \\ &\quad + f_3'(x) \cdot f_1(x) \cdot f_2(x) \cdot \prod_{i=4}^n f_i(x) \\ &\quad + f_{n-1}'(x) \cdot f_1(x) \cdots f_{n-2}(x) \cdot f_n(x) \\ &\quad + f_n'(x) \cdot f_1(x) \cdots f_{n-1}(x) \end{align*}$$

We’re adding up a bunch of terms, each of which is a derivative $f_i'(x)$ multiplied by all the other $f_j(x)$’s, where $j \neq i$. Landing this messy, notation-heavy plane, we can rewrite this long sum as

$$\frac{d}{dx} \prod_{i=1}^n f_i(x) = \sum_{i=1}^n f_i'(x) \cdot \frac{\prod_{j=1}^n f_j(x)}{f_i(x)} = \prod_{i=1}^n f_i(x) \cdot \sum_{i=1}^n \frac{f_i'(x)}{f_i(x)},$$

which is the solution we came for.

Method 2: Leveraging Logarithms

Mathematicians crave elegance, and Method 1 wasn’t that. We had to keep keen eyes out for patterns as they emerged and carefully distribute/rearrange many terms to get the solution. This second method will be much cleaner, but as many clever methods do, it begins with an unintuitive step.

Let $F(x) = \prod_{i=1}^n f_i(x)$. Taking the natural logarithm of both sides,

$$\ln(F(x)) = \ln\left(\prod_{i=1}^n f_i(x)\right) = \sum_{i=1}^n \ln(f_i(x)),$$

where the last equality follows from the fact that the logarithm of a product is equal to the sum of logarithms. The derivative of the natural logarithm is:

$$\frac{d}{dx}\ln(x) = \frac{1}{x}$$

Using chain rule to take the derivative of both sides of the above equality,

$$\begin{align*} \frac{d}{dx}\ln(F(x)) &= \frac{d}{dx}\sum_{i=1}^n \ln(f_i(x)) \\ \frac{1}{F(x)} \cdot \frac{dF}{dx} &= \sum_{i=1}^n \frac{1}{f_i(x)} \cdot f_i'(x) \\ \frac{dF}{dx} &= F(x) \cdot \sum_{i=1}^n \frac{f_i'(x)}{f_i(x)} \end{align*}$$

If we plug $F(x) = \prod_{i=1}^n f_i(x)$ back in, out pops the solution:

$$\frac{d}{dx} \prod_{i=1}^n f_i(x) = \prod_{i=1}^n f_i(x) \cdot \sum_{i=1}^n \frac{f_i'(x)}{f_i(x)}$$

Concluding Thoughts

Having initially stumbled upon Method 1, I was surprised that this derivative came out so beautifully in the end. Credit where it’s due: I adapted the elegant Method 2 from this post regarding infinite products. I have some technical notes at the end of the linked PDF about when Method 2 breaks down, though I believe Method 1 does not have any of these problems.

Hopefully this was fun (and if it wasn’t, I doubt you read this far 😉). This is one of many examples in mathematics of finding multiple solutions to the same problem and learning something different each time. Next time you emerge victorious from a tedious derivation, see if you can find a clever alternative!

Solving the Molecube by Reduction

Mon, 06 Jan 2020 13:00:00 -0700

My sister-in-law surprised me with a Molecube for Christmas, which combines the logic of a Sudoku puzzle with the mechanics of a Rubik’s cube. Each ball on the Molecube is one of nine colors, and the goal is to reconfigure a shuffled Molecube so each of its faces has all nine colors on it.

It turns out that solving the Molecube is a wonderful exercise in what computer scientists call reduction, which involves transforming a problem we don’t know how to solve into one that we do, solving that version, and then translating the solution back into the original problem. In this post, I’ll give a reader-friendly primer on reduction, outline a reduction from the Molecube to the Rubik’s cube, and then wrap up by solving the Molecube with a standard Rubik’s cube algorithm.

If you’re interested in solving the Molecube yourself (and perhaps you’re here looking for hints), I’ve created a worksheet you can use and will point out when you should skip ahead in the post so as to avoid spoiling this delightful puzzle.

Reduction: There and Back Again

Say we have a problem $A$ that we don’t know how to solve and another problem $B$ that we do know how to solve. Said another way, we have an “algorithm” that can answer any question asked in the form of problem $B$, but we have no such algorithm for problem $A$ questions. The whole idea of reduction is to:

Take a problem $A$ question (which we don’t know how to answer) and “transform” it into a problem $B$ question (which we do know how to answer).
Use the “algorithm” for problem $B$ questions to get an answer for our transformed problem $A$ question.
Translate the algorithm’s answer back into the context of problem $A$.

That’s it. We use a known algorithm (for problem $B$) as our workhorse for a new and unknown problem (problem $A$), and out pop solutions for our unknown problem! This powerful technique underlies almost all of theoretical computer science, giving us a tool to relate difficult problems to one another (as opposed to treating every new problem as something totally unique). I recently heard a high-profile professor in Computer Science claim that reduction is one of only two truly new ideas our discipline has ever contributed to science (though she called this idea “hierarchy”, with the other idea being “abstraction”).

A natural question to ask next would be if all problems can be tackled with reduction. Unfortunately, in practice, finding the right translation between a pair of problems (Step 1, above) can be prohibitively difficult. Reduction is easiest when the two problems seem to have an obvious relationship we can exploit, which brings us to the Molecube and the Rubik’s cube.

A Sudoku-Like Transformation

It’s difficult to overstate how much of the Molecube’s solution is given away in its advertising as “Sudoku + Rubik’s cube”. My solution will treat this equation quite literally, starting with a transformation that relies on a Sudoku puzzle. I found this transformation by asking two simple questions:

How are the Molecube and the Rubik’s cube similar?

They’re both 3x3x3 cubes, meaning they both have 6 faces, 8 corners, and 12 edges. This totals 26 balls (on the Molecube) or blocks (on the Rubik’s cube).
Their physical mechanics (spinning, twisting, etc.) are identical.
Their goals are, in a way, also identical: from a shuffled configuration, reach a goal configuration.

How are the Molecube and the Rubik’s cube different?

Their goal configurations are different: the Molecube wants one ball of each color on each face, while the Rubik’s cube wants each face to be all the same color.
There are nine colors on the Molecube, but only six on the Rubik’s cube.
A block on the Rubik’s cube has 1–3 colors (one for each face it touches), while a ball on the Molecube is the same color on all “sides”.

The similarities hint at a solution: though the goal configurations are different, the cubes' structures and mechanics are the same. So if I find a goal configuration for the Molecube, I can use the Rubik’s cube algorithm to handle all the tricky rearranging involved in actually getting there.

If that’s enough of a framework for you to attempt your own solution, feel free to download the worksheet I’ve created to help you visualize the Molecube as a Sudoku puzzle (with colors instead of numbers). You’ll want to stop reading here and come back once you’ve completed a goal configuration or if you’re stuck and need hints.

Speaking of hints, the best way to unlock this tricky Sudoku-like color puzzle (getting one color on each face) is to study the Molecube’s colors and structure. I asked myself the following questions (which culminated in the table at the top of the worksheet):

How does the Molecube fit nine colors on a cube with 26 balls?
How are the colors distributed over the different types of balls (centers, corners, and edges)?
Are there any patterns that appear when trying to place balls of a certain color so it appears on each face exactly once?

The answers to these questions are revealing. There are three balls of each color, with the important exception of Green, which only has two. Further, Green is the unique color that is on two corners. Red and Purple are each on three edges, and the remaining six colors (White, Black, Orange, Yellow, Light Blue, and Blue) are each on one center, one corner, and one edge. This information — after some careful thinking — reveals the patterns we need:

As in the Rubik’s cube, centers opposite one another are always opposite one another (i.e., Black is always opposite White, Orange is always opposite Yellow, and Blue is always opposite Light Blue).
The only way the two Green corners avoid being on the same face is if they’re opposite one another (e.g., upper-right-back and lower-left-front).
The only way the three Red (or Purple) edges avoid being on the same face is if a Red (or Purple) appears exactly once in each “middle band” (shown below, left). So each middle band contains exactly one Red and one Purple.
For any of the remaining colors, (e.g., Blue) there is a center of that color. This blocks the corner of that color from being in the same “layer”, so the corner must go in the layer opposite the center (shown below, right). Once the position of the corner is fixed, there is only one position the edge of that color can go.

Using these rules and some trial and error, I found the solution shown below, also detailed in the worksheet solution. (An interesting aside: I don’t know if this is the only solution, but any solution works for the reduction). To relate the Molecube solution to the Rubik’s cube solution, we simply treat each 3×3 face on the Molecube as a “color” on the Rubik’s cube. (For example, the Black ball at the top-left of the White-center face becomes the Red-White-Green block on the Rubik’s cube). This completes the transformation step of the reduction.

Rubik-ing the Molecube

With the difficult transformation step out of the way, the rest of the reduction is easy. You can solve any shuffled Molecube just like you would a Rubik’s cube (assuming you know how to do that), but instead of aiming to have faces with all the same color, you aim to build the goal configuration we got from the transformation (above, left). To make this easier for me to visualize, I made a 3D rendering of my solution (front view on the left, back view on the right).

I was pleasantly surprised at how much Rubik’s cube muscle memory I still had from speedcubing in junior high (though if you need a refresher, I found this tutorial helpful). Interestingly, some of the steps in the Rubik’s cube “algorithm” are unnecessary for the Molecube. Remember the earlier observation that the Molecube’s balls are each a single color while the Rubik’s cube blocks can have 1–3 colors each? This means that the Molecube doesn’t care if its balls are rotated “in place”, though this is a problem for the Rubik’s cube (see the example below). So any steps in the Rubik’s cube algorithm that are meant to fix things like this can be skipped entirely.

But Does It Work?

To show that the reduction approach not only works but is also reasonably fast, here’s me solving the Molecube in just under 2 minutes.

There are a lot of Rubik’s cube-inspired puzzles these days (for example, the Ghost Cube and the Pyraminx). But solving the Molecube by reduction makes me wonder just how many of these new puzzles share a similar relationship to the original Rubik’s cube. If these relationships exist, we’d find that these new puzzles aren’t really new at all; they’re just an old puzzle we know how to solve, but with new names and nice packaging.

The Three Academics: A Joke

Tue, 30 Jul 2019 09:15:00 -0700

Mathematicians really care about correctness. We believe unspoken assumptions are dangerous. We think vague language is confusing, and saying more or less than we mean is grounds for misunderstanding. We want terms to be clearly defined, and for points to follow logically, like links in an unbreakable chain of truth. But sometimes people say things that are completely reasonable and yet we still feel the need to set them straight:

This is my first contribution to the world of webcomics, a medium I’ve long admired for being able to deliver witty, weird reflections on life in simple packages. Giving credit where it’s due, this “joke” is adapted from The Curious Case of the Dog in the Night-Time by Mark Haddon and the art style takes after poorlydrawnlines by Reza Farazmand. In living out generous making, this comic is available as a high resolution PNG or PDF for you to use as you’d like. (Just keep my authorship in the corner intact, please!)

And hey, to the economists out there: if you’ve got any good roasts of mathematicians, I’m all ears.