Relational Thinking in J

According to Aaron Hsu, the starting point for APL programming is the relational model.

I’m mixed on this, because I don’t think J has a natural concept of table, actually. Moreover, it seems like your code gets littered with head {. and constant-powered { lookups if you do represent data in a tabular format, with heterogeneous rows. I could be wrong, but it seems to work better in general when you have homogeneous arrays. Creating weird packets of data like we do in other languages just doesn’t seem to be the ticket here.

Suppose you design a table like this:

You will probably wind up using 0 { or 1 { to take it apart to do different things with the different columns. So I would probably build this table in J like so:

   developers =. 'alice'; 'bob'; 'calvin'; 'delilah'; 'ellen'
   devemails =. 'a@example.com'; 'b@example.com'; 'c@example.com'; 'd@example.com'; 'e@example.com'

This is maybe a column-oriented view of the world. You can recover the table pretty easily though:

   developers ,. devemails
┌───────┬─────────────┐
│alice  │a@example.com│
├───────┼─────────────┤
│bob    │b@example.com│
├───────┼─────────────┤
│calvin │c@example.com│
├───────┼─────────────┤
│delilah│d@example.com│
├───────┼─────────────┤
│ellen  │e@example.com│
└───────┴─────────────┘

Projection is sort of obvious now, you have to choose the columns you want because you don’t have the table, as it were. Selection isn’t so bad; you are going to filter on a certain column and apply that filter on the other column. Let’s find the developers with an “a” in their name:

   developers #~ 'a' e."1> developers
┌─────┬──────┬───────┐
│alice│calvin│delilah│
└─────┴──────┴───────┘

The same selection works on the other column, and you can still stitch together columns to make a table:

   devemails #~ 'a' e."1> developers
┌─────────────┬─────────────┬─────────────┐
│a@example.com│c@example.com│d@example.com│
└─────────────┴─────────────┴─────────────┘

   (#&developers ,. #&devemails) 'a' e."1> developers
┌───────┬─────────────┐
│alice  │a@example.com│
├───────┼─────────────┤
│calvin │c@example.com│
├───────┼─────────────┤
│delilah│d@example.com│
└───────┴─────────────┘

Joins work by doing lookups by index. Let’s introduce another table:

Following the earlier example we get this:

   projects =. 'alphago'; 'bitbucket'; 'cafeteria'; 'diffie'; 'entryway'; 'finality'; 'grace'; 'homelab'
   projdevs =. 'calvin'; 'ellen'; 'delilah'; 'alice'; 'bob'; 'alice'; 'delilah'; 'calvin'

   projects ,. projdevs
┌─────────┬───────┐
│alphago  │calvin │
├─────────┼───────┤
│bitbucket│ellen  │
├─────────┼───────┤
│cafeteria│delilah│
├─────────┼───────┤
│diffie   │alice  │
├─────────┼───────┤
│entryway │bob    │
├─────────┼───────┤
│finality │alice  │
├─────────┼───────┤
│grace    │delilah│
├─────────┼───────┤
│homelab  │calvin │
└─────────┴───────┘

We can find the email of the developer for each project like so:

   (developers i. projdevs) { devemails

┌─────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┐
│c@example.com│e@example.com│d@example.com│a@example.com│b@example.com│a@example.com│d@example.com│c@example.com│
└─────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┘

This might be easier to read as a table, so let’s do that:

   projects ,. projdevs ,. devemails {~ developers i. projdevs
┌─────────┬───────┬─────────────┐
│alphago  │calvin │c@example.com│
├─────────┼───────┼─────────────┤
│bitbucket│ellen  │e@example.com│
├─────────┼───────┼─────────────┤
│cafeteria│delilah│d@example.com│
├─────────┼───────┼─────────────┤
│diffie   │alice  │a@example.com│
├─────────┼───────┼─────────────┤
│entryway │bob    │b@example.com│
├─────────┼───────┼─────────────┤
│finality │alice  │a@example.com│
├─────────┼───────┼─────────────┤
│grace    │delilah│d@example.com│
├─────────┼───────┼─────────────┤
│homelab  │calvin │c@example.com│
└─────────┴───────┴─────────────┘

So there’s some basic relational-type stuff in J. Is this the right approach? I don’t know.

Edit: Notes from the chatroom from elcaro:

You want to be a little bit careful when dealing with boxes, because unboxing a boxed array will create fills. When you do 'a' e."1> developers, your right arg is a 5×7 character vector

   quote"1 > developers 
'alice  '
'bob    '
'calvin '
'delilah'
'ellen  '    

If you looked for developers with a Space in them, you’d match all except ‘delilah’. You can unbox each, and so compare each unbox string by itself ('a' e. ])@> developers 1 0 1 1 0

Which is fine, but might be hard to put into a larger tacit expression, as you need to explicitly refer to left arg in that inner expression. Something else you can do is use Spread (S:) or Level-At (L:) where appropriate. I don’t use these much, but S:0 often pairs nicely with e. or E., where you want to match some unboxed string to a list of boxed strings

'a' e.S:0 developers 1 0 1 1 0

This new operator is often easier to put into a larger tacit expression (if desiered) Spread and Level are a little like the Depth modifier in BQN ⚇ (and proposed in APL ⍥) in that it modifies operators to perform on nested data. The Nuvoc pages are a little lacking, but play with them and you’ll get a hang of what they do Spread - NuVoc Leval-At - NuVoc Depth (operator) - APL Wiki Depth - BQN docs

Oh one more thing… you can do

(#&developers ,. #&devemails) 1 0 1 1 0

as

(developers ,. devemails) #~ 1 0 1 1 0

May 28, 2025 j

My thoughts on the CES Letter.

My friend Bill recommended I read CES Letter. I found it pretty hard to put down, and read the whole thing over the last couple days. I thought it was worth reflecting on the ideas in it from the point of view of a Baha’i.

I haven’t mentioned it on my blog yet for various reasons, but I declared myself a Baha’i in October 2023. So I now belong to a Faith about the size of Mormonism (although much smaller in the US). I have been met with quite a bit of understanding and maybe a little puzzlement. But this seems like a good context to talk about it, because any faith should be able to withstand the kinds of questions Jeremy Runnells asks of his former faith. Why not start with the controversial stuff? At least you won’t think I’m being fatuous.

Before going too far down this road, I want to make it clear I’m not going to bag on LDS believers. Baha’u’llah says “Consort with the followers of all religions in a spirit of friendliness and fellowship.” Reading the CES Letter, I felt a lot of sympathy for Jeremy and others like him: good and decent people trying to adhere to God’s wishes, whom I have no doubt are worthy of and will receive an ample share of God’s blessings. And furthermore, I’m just a random Baha’i and a guy who has read some books; I don’t speak from authority about anything, let alone the Baha’i Faith.

The CES Letter is concerned with a number of topics: the verity of the foundational texts, the legitimacy of Joseph Smith Jr and the line of authority starting with him, the legitimacy of his witnesses, certain difficult or troubling teachings, and the Church’s attitudes towards historicity, fact, belief, emotion and the general reconciliation of reason with, let’s say, shortcomings in those categories.

There are analogs in some of these areas to the questions that are asked of the Baha’i Faith. In Mormonism, the question about the texts has to do with their historicity and legitimacy. In the Baha’i Faith, we have a large number of writings that are untranslated. Both religions have gone through succession crises and wound up with basically one real movement and a few tiny groups of sectarians. Neither is especially progressive about LGBTQ issues, and both have a kind of membership status that can be imperiled by transgressing certain behavioral standards including this. Both have a concept of infallibility, a word that makes my skin crawl a bit and probably yours too.

Probably the most basic difference is that the Baha’i Faith requires you to figure things out on your own. While you may be born into a Baha’i family and may get a Baha’i education, in the end, it’s completely your personal decision. When you declare, at least in the US, you fill out an online form; if you withdraw, you click a button on the same site. You’re not going to get harassed about it. You can attend Baha’i gatherings as a Baha’i or just a friend; declaring and undeclaring does not imperil this. Nothing is really at stake if you reject the Faith. The benefits of being a declared Baha’i are simply that you can donate money, participate in the electoral process and take part in the internal discussion part of the 19-day meeting (“feast”). Nobody believes that you are going to hell or otherwise going to suffer some kind of penalty in the afterlife. Similarly, the Baha’i behavior standards are for Baha’is. If you have not declared, they obviously do not apply to you, and nobody will think less of you for drinking or whatever. Anyone can use the Baha’i prayers, read the Baha’i books, believe in any part of it or none of it; it is for everyone equally.

Based on the CES Letter, it sounds like for Mormonism, the important thing is that you feel it is true, and this is called a testimony. Based on this, you’re supposed to accept everything. The outcome for Jeremy was extremely difficult. It’s not a good look for the LDS church, in my opinion.

Now, infallibility gives me and most Americans hives because we find it impossible to imagine that someone could never have made a mistake about anything. The combination of infallibility with two known-invalid translations and a third whose source is not available kind of beggars belief. For Baha’is, the combination of infallibility with a large corpus of untranslated text at first sounds like a minefield. I have only read a small portion of what is available, because despite it being a fraction of what exists, it’s still enormous. In doing this reading, what I came to realize is that generally when something is said and I have trouble with it, when I ponder it, I come to see how it fits with everything else. Though there is a vast amount of writing, the majority of it closes in on the same themes repeatedly: the unity of mankind, the unity of religion, the unity of the world. It is tremendously optimistic. Having read as much as I have, my fears about what remains untranslated have reduced a lot. Baha’u’llah didn’t sit around holding forth on every possible topic. He talked about the same things repeatedly with many different people, using different kinds of language.

In the light of unity, I see infallibility as primarily serving the role of bringing about unity by proscribing intense, schism-causing debate. In this perspective, Baha’u’llah gave leadership to Abdu’l-Baha’ and declared his infallibility so that there would not be schism from different people trying to usurp the religion. (Many tried anyway.) The same story repeats with Shoghi Effendi. In neither case were they empowered to add new things to the religion, only to explain. Each of these were small succession crises in comparison to Shoghi Effendi’s death and the formation of the UHJ. But even there, enough groundwork had been laid that the vast majority of people came along to the Baha’i Faith we have today, and only a small splinter group was created (mostly living in Roswell, NM, of all places). There are only a few examples people have found where Abdu’l-Baha’ appears to say something contrary to science; the most prominent one is probably this one:

Question: What will be the food of the united people? Answer: As humanity progresses, meat will be used less and less, for the teeth of man are not carnivorous. For example, the lion is endowed with carnivorous teeth, which are intended for meat, and if meat be not found, the lion starves. The lion cannot graze; its teeth are of different shape. The digestive system of the lion is such that it cannot receive nourishment save through meat. The eagle has a crooked beak, the lower part shorter than the upper. It cannot pick up grain; it cannot graze; therefore, it is compelled to partake of meat. The domestic animals have herbivorous teeth formed to cut grass, which is their fodder. The human teeth, the molars, are formed to grind grain. The front teeth, the incisors, are for fruits, etc. It is, therefore, quite apparent according to the implements for eating that man’s food is intended to be grain and not meat. When mankind is more fully developed, the eating of meat will gradually cease.

On the one hand, we have an evolutionary explanation for why we have incisors: tearing meat, probably. But the question isn’t about where we have come from, it’s about where we are going, and virtually everyone can agree that eating more meat is probably worse for us. Evolution tells us where you came from, it doesn’t necessarily tell you where you’re going; giant pandas have incisors too, but only eat bamboo.

In the Baha’i Faith, science and religion are complimentary because one tells you where you have been and the other where you are going; one tells you how things are and the other tells you why. As Baha’u’llah says: “Knowledge is as wings to man’s life, and a ladder for his ascent. Its acquisition is incumbent upon everyone.” While there is an emotional, experiential dimension to the Baha’i Faith, it is not to override reason. This does not appear to be the case in the LDS church.

On the question of LGBTQ issues, we have about the same in the Baha’i Faith as in the other major Abrahamic religions: a definition of marriage as being between a man and a woman and that sexual relations are to be limited to this union. Baha’u’llah made a very oblique mention of pedophilia in the book of laws; Shoghi Effendi expanded this to homosexual relations. I can’t reconcile myself to Shoghi Effendi’s interpretation, but he certainly did not have the power to add something new to the writings or abrogate something completely in order to arrive at another solution. In the interest of unity, I can only acquiesce to it. There are LGBTQ Baha’is, as well as Muslims, Catholics and Jews; I think Baha’u’llah certainly did not want us to construct our identities on the basis of our preferences, and this is simply one of those tests for us, especially Americans who are accustomed to equating the satisfaction of our desires with the good. The matter is, in my opinion, somewhat overstated, because the Faith does not really permit us to categorize people into “good” and “bad” categories (or “in-group”/“out-group” or any other dichotomy) on any basis. If you are discriminated against by a Baha’i, they have failed to fulfill their obligations to Baha’u’llah to create unity.

Unlike Mormonism, we simply don’t have questions about the accuracy or authenticity of our religious texts. Also unlike Mormonism, we get to have the benefit of most other religion’s texts. The validity of our translations are sometimes questioned. The Arabic of The Báb and Baha’u’llah were not exactly normative; Modern Standard Arabic wasn’t yet really formalized and their styles are idiosyncratic and have a lot of Persian influence. The Báb’s writings were only intended to last a short period of time and are written in a very dense style for people highly acquainted with the Qur’an; translating the bulk of them into English has not been a high priority. Those interested in knowing more about His writings should probably read Gate of the Heart. I don’t know the precise reason why much of Baha’u’llah’s writings haven’t yet been translated. I suspect that the reasons are simply because A) there is so much to translate, B) Shoghi Effendi translated what he considered important for us, and there is a general trust in his decisions, C) not much trust in our ability to do an equally good job, D) the funds are better spent in other ways, and E) a lot of Baha’u’llah’s writings are direct correspondence with believers and others, so may be repetitive in toto or not particularly general. These are just my speculations.

The historicity claims of the LDS church are pretty integral to the entire enterprise. If Joseph Smith didn’t really find hidden tablets, if the tablets aren’t true, then the rest of his revelation is probably false, which is what provides the impetus for their being an LDS church at all. Would the Book of Mormon be worth reading if it were an out-and-out forgery? The context surrounding it makes it difficult for non-Mormons to take seriously, and the Book of Abraham and Kinderhook Plates are obviously forged. Is there still value in reading and studying these books? I’m not in a position to say. Much of the Hebrew Bible is either unverifiable or false from a historical perspective; the majority of Jews don’t see a problem with this and find that the ideas are still useful and worth studying. Most of Jesus’s words are in the form of parables, which means we know that their “truth value” in the sense of logic is false, but their “truth value” in the sense of spiritual teachings is quite large. On the other hand, it is the truth value of the LDS’s teachings that gave the LDS church racist teachings, promoted polygamy and conversion therapy, several of which were abrogated only recently.

The Baha’i Faith has some historicity problems of another sort. The obvious one is that if all religions are one, why are they so different from one another? The Baha’i perspective is that there are core teachings that are the same for all religions, which have been promoted by the Manifestations of God in all eras and all regions; the details differ because the locality and era might demand different emphasis, or perhaps just due to the passage of time and meddling by religious authorities (a favorite target of Baha’u’llah’s). This seems pretty workable for the Abrahamic faiths that preceded it but raises problems about Eastern religions that do not have obvious solutions. Consider this question about Confucius and Buddha:

Buddha also established a new religion and Confucius renewed the ancient conduct and morals, but the original precepts have been entirely changed and their followers no longer adhere to the original pattern of belief and worship. The founder of Buddhism was a precious Being Who established the oneness of God, but later His original precepts were gradually forgotten and displaced by primitive customs and rituals, until in the end it led to the worship of statues and images.

It’s quite difficult to look at the Buddhism we have today and see how Buddha could possibly have been talking about the oneness of God when there are no Buddhisms today that talk about God. Moojan Momen wrote a lengthy article about the interface between the Baha’i Faith and Buddhism centered around the 8-fold path. But it’s unlikely that a Buddhist would be interested in hearing from us that they were originally monotheists. But Buddhism is so old and the original writings long gone, so what we have here from Abdu’l-Baha’ is basically untestable. His interpretation of Buddhism as worshipping images would be offensive to Western Buddhists. But I’m not in a position to judge whether it is true elsewhere; my sense is that Buddhism in the West is rather different from Buddhism in the East, where people actually visit Buddhist shrines and have them in their homes. But back to the point.

Faced with a particular instance of an infallible person saying something at least untestable but perhaps false, I could respond by just giving up on it. I could respond to the unacceptance of gay marriage by giving up on the Faith. I didn’t though.

1. There is not some slightly-tweaked form of the Faith out there that would resolve all these problems; there’s one Baha’i Faith.
2. If there were, how would siding with some splinter group help working towards unity?
3. There is not a place in the Faith where it says it is my job to judge people on the basis of their religion or sexuality or anything else. On the contrary, I am to accept everyone, show kindness to everyone, show violence to no one, speak ill of no one, etc.
4. What we are trying to build is bigger; quibbling about this or that element creates more disunity rather than more unity.
5. I can accept and show love to anyone, even if I can’t change doctrine.

Whether Abdu’l-Baha’ is right about the historical Buddha, we will probably never know, but I don’t think it undermines my religion. We have what we have today. Building bridges between Buddhism and the Baha’i Faith, that is something that matters today, moreover between Buddhists and Baha’is. Whatever their beliefs are based on, I still think we can learn something from them today. And it’s the same with the LDS church. We can learn things about God, humanity, how to be a better person and so forth from Mormons, from Buddhists, from LGBTQ people, from atheists. And we should, and we have to.

So this my interpretation of infallibility: that it is more about unity than about being right about everything. Much like in a marriage, sometimes you have to set aside the question of who is right in order to be happy and have peace. Hopefully not all the time. The marriage is more important. It can be, anyway, ideally. But also like a marriage, it’s hard to rebuild trust after a lot of lying. After having gone through the CES Letter, it feels like there is a little too much fabrication.

Overall, the CES letter is a great read. I recommend it to you whether you are an LDS member or not. I only disagree with one idea: “Each religion has believers who believe that their spiritual experiences are more authentic and powerful than those of the adherents of other religions. They cannot all be right together, if at all.”

May 25, 2025 religion

Internationalization Puzzles in J, Day One

I just found out about these i18n puzzles and figured I’d take a crack at one in J. The first one is pretty easy. I’m also trying to apply my learning about linear algebra to the domain.

The crux of this puzzle is the following observation about a string:

• For SMS, the byte count matters, and must be under 160 bytes
• For Twitter, the character count matters, and must be under 140 characters

The input format is just a list of strings, and your mission is to calculate the cost. I glanced over this at first and made a mistake in my zeal to apply some really rudimentary linear algebra. Basically, I thought, let’s convert the input into a big matrix, we’ll have a byte count $b_1$ and a unicode length $u_1$ for each, and this will become our input matrix:

${\displaystyle \left[\begin{array}{cc}{b}_1<160& u_1<140\\ b_2<160& u_2<140\\ b_3<160& u_3<140\end{array}\right]}$

This will become a matrix of 0s and 1s, which we can then just take the dot product by the costs matrix:

${\displaystyle \left[\begin{array}{c}\mathrm{\$}0.07\\ \mathrm{\$}0.11\end{array}\right]}$

The entire problem should reduce to something like this:

${\displaystyle \left[\begin{array}{cc}0& 0\\ 1& 1\\ 0& 1\\ 1& 0\end{array}\right]\cdot \left[\begin{array}{c}\mathrm{\$}0.11\\ \mathrm{\$}0.07\end{array}\right]=\left[\begin{array}{c}\mathrm{\$}0.00\\ \mathrm{\$}0.18\\ \mathrm{\$}0.07\\ \mathrm{\$}0.11\end{array}\right]}$

If I had done this step on paper or something, I would probably have figured out the mistake, but I didn’t until later.

Let’s translate that to J. First we need to read the file, which will be using input =. cutLF fread filename. This gives us boxed strings, which is fine. Now we need to use # to get the length and ucpcount to count Unicode characters. We can throw these together as a train with , to get both at once:

   (# , ucpcount) each input
┌───────┬───────┬───────┬───────┐
│162 143│138 136│253 140│147 141│
└───────┴───────┴───────┴───────┘

These are the very values in the problem page, so this appears to be on the right track.

Then I hit a little snag with trains, because I wanted to write it like (160>:# , 140>:ucpcount) but this does not do what it feels like it should, on account of the strict left-to-right order. So I wrote it like so instead:

   >((160>:#) , 140>:ucpcount) each input
┌───┬───┬───┬───┐
│0 0│1 1│0 1│1 0│
└───┴───┴───┴───┘

   >((160>:#) , 140>:ucpcount) each input
0 0
1 1
0 1
1 0

Now we have exactly the matrix I expected to have, so let’s try the dot product:

   (11 7) +/ .* |: >((160>:#) , 140>:ucpcount) each input
0 18 7 11

This is supposed to be 0 13 7 11? Oh right, in my excitement I forgot that I need to discount when they’re using both SMS and Twitter. I thought about this for a second and thought, I would really like to be able to index an array by another array. I’m not sure how that would work. But I also remember reading about a trick where you convert the two-dimensional index into a scalar by using encode #.. So instead of having a 2x2 table, we just have an array of length 4. In other words, 0 0 = 0, 0 1 = 1, 1 0 = 2, 1 1 = 3. Then I can encode the prices as 0 7 11 13, the price for nothing, a Tweet, an SMS, and both.

   (0 7 11 13) {~ 2 #. > ((160>:#) , 140>:ucpcount) each input
0 13 7 11

Now we can just make the entire solution:

   +/ (0 7 11 13) {~ 2 #. > ((160>:#) , 140>:ucpcount) each (cutLF fread'~/Downloads/test-input.txt')
31

And this solves the puzzle.

Edit: the helpful people on The APL Farm provided some advice. For starters, Time Melon points out that #. has a default left argument of 2, so we can simply remove the 2 there, and the parentheses around the prices can be removed, yielding this improvement:

   +/ 0 7 11 13 {~ #. > ((160>:#) , 140>:ucpcount) each input
31

Elcaro points out that each is creating boxes I am then removing, so we can simplify to this:

   +/ 0 7 11 13 {~ #. ((160>:#) , 140>:ucpcount) every input
31

or this; I’m undecided but leaning towards the shorter one because I didn’t realize every was a thing:

    +/ 0 7 11 13 {~ #. ((160>:#) , 140>:ucpcount) &> input
31

Elcaro also noticed that I’m missing out on the obvious fact that >: is repeated inside the major transformation, so we can simplify it further to this:

   +/ 0 7 11 13 {~ #. (160 140 >: #,ucpcount) &> input
31

   NB. Or directly
   +/ 0 7 11 13 {~ #. (160 140 >: #,ucpcount) &> cutLF fread '~/Downloads/test-input.txt'
31 

And this appears to me to be the final form!

Edit: Elcaro makes another suggestion, pointing out that cutLF is not that different from <;._2, and so we can actually remove the boxing altogether and simplify the solution slightly further to this:

   +/ 0 7 11 13 {~ #. (160 140 >: #,ucpcount);._2 input
31

May 20, 2025 j

What is proof to a non-mathematician?

I was talking to Alex this morning and the topic of geometric proof came up. This caused me to spend some time idly thinking about proof. I’ve written about this before but let me expand on it kind of dreamily.

I learned about Coq from a seminar taught by another friend of mine, Tyler Cecil. A key idea I learned from him is audience. The nice and beautiful thing about using a theorem prover like Coq or Lean is that it has the same excruciating standards for everyone. It entices the programmer by making you think that proving something is about the same as writing a program. This is formally true, thanks to the Curry-Howard Isomorphism but the skills needed to write a proof and the skills needed to write a program are curiously different. You wind up needing to know a bit of both to use a theorem prover well.

My friend Randy Van Why explained to me why mathematicians do not especially love theorem provers. In short, they are not solely interested in the truth-value of a proposition. They want to understand why a theorem is true. Input to a theorem prover tells you that a theorem is true, and in many cases it can do a proof-by-exhaustion that human could never pull off. But exhaustively testing every possibility to find out that the four-color theorem is true is not the same as the elegant proof of the five-color theorem. We want to learn something from a proof in addition to the truth value.

Parenthetically, Randy also informed me that the majority of mathematical communication happens over the chalkboard, not paper. So don’t feel bad if you have trouble self-studying math, because mathematicians mostly don’t.

One thing computer scientists like about theorem provers is that they can be restricted to constructive proof. This means certain avenues of proof are unavailable, and some things cannot be proven, but what you can prove, you can also construct—or compute.

However, when I am proving something on the side for fun, the only person I need to convince is myself. Kind of like playing chess with yourself, you have to train yourself a little in how to do it. However, the game is a bit more fun. Often if you don’t know how to do a proof from a book, it’s because you have forgotten an important definition from earlier. Sometimes you misunderstand the situation and make a bad proof you think is good. The time you spend shouting at the page, “this is obvious! why can’t I prove it?” seems to be time well-spent.

Here’s an example from the book No Bullshit Guide to Linear Algebra:

Verify that $|||| = 1$.

Well, we have a few definitions we’ll need to consult:

$|| || = || (v_x,v_y) || =$

and

$=$

Knowing these, we can expand like so:

${\displaystyle \begin{array}{rlr}{\displaystyle \mathrm{\mid }\mathrm{\mid }\widehat{v}\mathrm{\mid }\mathrm{\mid }}& {\displaystyle =}& {\displaystyle \mid \mid \frac{\vec{v}}{\mathrm{\mid }\mathrm{\mid }\vec{v}\mathrm{\mid }\mathrm{\mid }}\mid \mid }\\ {\displaystyle }& {\displaystyle =}& {\displaystyle \mid \mid (\frac{v_x}{\mathrm{\mid }\mathrm{\mid }\vec{v}\mathrm{\mid }\mathrm{\mid }},\frac{v_y}{\mathrm{\mid }\mathrm{\mid }\vec{v}\mathrm{\mid }\mathrm{\mid }})\mid \mid }\\ {\displaystyle }& {\displaystyle =}& {\displaystyle \sqrt{{\left(\frac{v_x}{\mathrm{\mid }\mathrm{\mid }\vec{v}\mathrm{\mid }\mathrm{\mid }}\right)}^2+{\left(\frac{v_y}{\mathrm{\mid }\mathrm{\mid }\vec{v}\mathrm{\mid }\mathrm{\mid }}\right)}^2}}\\ {\displaystyle }& {\displaystyle =}& {\displaystyle \sqrt{{\left(\frac{v_x}{\sqrt{v_x^2+v_y^2}}\right)}^2+{\left(\frac{v_y}{\sqrt{v_x^2+v_y^2}}\right)}^2}}\\ {\displaystyle }& {\displaystyle =}& {\displaystyle \sqrt{\frac{v_x^2}{v_x^2+v_y^2}+\frac{v_y^2}{v_x^2+v_y^2}}}\\ {\displaystyle }& {\displaystyle =}& {\displaystyle \sqrt{\frac{v_x^2+v_y^2}{v_x^2+v_y^2}}}\\ {\displaystyle }& {\displaystyle =}& {\displaystyle \sqrt{1}}\\ {\displaystyle }& {\displaystyle =}& {\displaystyle 1}\end{array}}$

This is a pretty simple example of a direct proof. In my opinion, this is enough detail. Someone newer to this subject matter might be confused by what happened between step 1 and 2, or someone with poor algebra might find the whole thing quite confusing. On the other hand, a more experienced mathematician would probably skip the penultimate step, or maybe even the step before that. The audience matters. But even if it’s just for you, I think doing these kinds of verifications is good for your understanding of the material.

May 14, 2025

Basic Stats in J

My son’s homework the other day involved the basic statistical functions of mean, median, mode and range. One of the most classic array language functions is, of course, calculating the mean, which is famously three functions: mean =. +/ % # in J.

The range is also pretty simple; you just want the difference between the largest and smallest values in the input, so you get range =. >./ - <./ in J.

The next two are not as clean. I’ll show you what I came up with and then we can compare to what’s going on in the stats/base library.

Median

Let’s start with the median. The idea is you take the middle value from the sorted list. “Middle value” makes I don’t see an obvious trick for finding the middle value in J. I see basically two ways to do it: 1) calculate the midpoint and index it, or 2) use a recursive grossness to tear bits of the start and end. The latter feels more like a functional programming solution with linked lists, so I’m guessing the more array-ish way of doing it is to calculate. But there’s a second problem, which maybe the dear reader has already seen but I had to actually get halfway there before I realized, which is that even length arrays do not have a value at the midpoint. This invites some overthinking.

My son’s book suggests that in this situation you average the two midmost values. On the one hand, it feels to me like the whole point of the median is that it is a value from the set. On the other hand, only having a median 50% of the time feels unsatisfactory. Having two results for median 50% of the time also seems bad. I checked Wikipedia, and they give a definition as a partial function:

if $n$ is odd, $(x) = x_{(n+1)/2}$ if $n$ is even, $(x) =$

This suggests a direct translation that would involve using agenda @. on the divisibility of the length. Another approach would be to always get two values and then if they happen to be the same, average them anyway, which feels like less work to me. If the list is odd-length, we should obtain a midpoint with a half index in it, as in a list of length 7 producing a midpoint of 3.5. We can subtract 0.5 from that to get the actual index of 3. However, if the list has a length of 8, we will get a midpoint of 4. Subtracting 0.5 from that will produce 3.5. If we take the floor and ceiling of this, we will get 3 and 4, which are the indices of the values around the middle. So it feels like the right approach here is to find the midpoint minus half, take the floor and ceiling of that, take the values at those indexes in the sorted list and average them. Which is a lot of words to say “find the median” but seems like the only way, going down this road. This yields the slightly unwieldy verb:

median =. {{ (+/ % #) q {~ (<.,>.) 0.5-~ -: # q =. /:~ y }}

So the idea here is take the argument, sort it, call it q because we’re going to need it again later, get the length of it, half it, take 0.5 from it, get the floor and ceiling of that number, take those from q, then average them. I’m not in love with this. Let’s see what we have in the stats/basic file:

min=: <./
max=: >./
midpt=: -:@<:@#
median=: -:@(+/)@((<. , >.)@midpt { /:~)

This is an interesting piece of code. We can see some similarity, though defining max/min and not using them is a choice. For midpoint, we have half of count minus one, which saves a few constants from my method. Same trick for getting the floor and ceiling in one: (<. , >.). Nice compositional style here, no need for the actual variables. Also no need for a real average, we just add up the two values and cut in half. All in all, this solution is shorter and makes much better use of the primitives than mine, which isn’t too surprising, and manages a tacit style nicely. I’m not sure how much value is obtained by moving midpoint out though.

Tentative conclusion: pay attention to constants like 1/2. If you know you have two items, you can use halve -:.

Mode

Now we have a real bummer: the mode. The mode is defined as the most frequent value in the set. We can safely assume that this is going to return a list of results because there can be many items with the most frequent value. I want to utilize key /.. as in a post from a few days ago to get the histogram, basically, of the input, and then we’ll need to utilize the two parts separately: find the largest count, and then use that to lookup the values with the largest count. For some reason I’m picturing this happening inside a verb that gets called dyadically, with the counts on the left and the groups of identical values on the right. It will be very interesting to see how this compares to the method in the stats/basics package.

So first, let’s get the counts with the values:

    (#,{.)/.~ q =. 9 1 8 16 42 9 1 2 8 7 7 5 4
2 9
2 1
2 8
1 16
1 42
1 2
2 7
1 5
1 4

Now this is slightly inconvenient, because I would really like to have the counts on one side and the values on the other so I can use insert / to put a function between them. We wind up needing to transpose this intermediate so we get the counts in the first entry and the values in the second:

   |: (#,{.)/.~ q
2 2 2 1 1 1 2 1 1
9 1 8 16 42 2 7 5 4

Now we need a function to find the greatest, which is simply >./. Now we have a selection problem, which demands filtering the input, which is a job for copy #. So we test for where the count on the left is greatest, find occurrences of that value in the left, and filter with copy against the right, and the overall solution is this:

   mode =. {{ (([ = >./@[) # ])/|: (#,{.)/.~ y }}
   mode q
9 1 8 7

This could probably be simplified; we could invert copy and remove some parens, for instance. I’m not entirely sure that we need to transpose the array.

Surprisingly, stats/base does not contain a definition for mode, but on the wiki I found this one: mode=: ~. {~ [: (i. >./) #/.~ which only returns the first value. But let’s look at how it works. First, we calculate the count over the self-groups, which gets us the frequencies in the histogram (but not their values). Then we have a capped fork. I don’t find these especially readable yet. Here it is now clear that we are finding the index of the largest value within the frequencies. Then we are inverting take and passing the nub of the original input to the right of take and the index of the largest value to the left. However, the use of i. here guarantees we get one result, no matter how many modes our set has. Thus, we get a pretty incomplete answer on our sample input:

   mode =. ~. {~ [: (i. >./) #/.~
   mode q
9

However, we know the answer we want will have #~ in the position where {~ is, and we know that we should have = where we have i., so let’s try that:

   mode =. ~. #~ [: (= >./) #/.~
   mode q
9 1 8 7

So that appears to have worked, which surprises me a little.

The cap here has actually done us a lot of good because not only has it saved us from writing @:, it has also saved us from quite a few parentheses, as this demonstrates:

   (~. #~ [: (= >./) #/.~) q
9 1 8 7
   (~. #~ (= >./) @: #/.~) q
9 1 8 16 42 2 7 5 4
   (~. #~ (= >./) @: (#/.~)) q
9 1 8 7

Tentative conclusion: /.. key dyad is very cute, but it’s easier to work with one piece of data at a time. Get better at capped forks, since they can be clearer than at @: when @: would require more parentheses. Forks are helpful.

May 8, 2025 j math

Dracula

We had to download a new app to watch the Superbowl back in February, Tubi. For some reason I was thinking about vampires back then, so I decided to watch the Hammer trilogy, The Vampire Lovers, Lust for a Vampire and Twins of Evil, which were all available on there, and then it recommended me another one, Vampyres, but I wouldn’t follow the IMDB link on that one if you’re at work. Somewhere in the middle I also watched Blood from the Mummy’s Tomb. These flicks are somewhere between horror and pornography. I’m not sure what happened, I think my brother had loved Hammer films when he was young, so they were kind of always on my to-watch list and I finally got around to a few, although I don’t think these are the ones he was endorsing exactly. Anyway, it was a vampire kick, these things happen. The movies are neither so great nor so bad that they deserve many words. But not ultimately satisfied, I decided I would watch a great vampire movie from my youth: Bram Stoker’s Dracula.

This should be a watershed movie. It has great effects, many great actors, great sets and was shot right around the time in my life when I feel like movies were really getting good. A certain scene near the beginning also activated certain neurons in your teenage author, the first movie that really imprinted on me for having done so. I had fond memories of this movie, so when I rewatched it with my wife, I was kind of awestruck at how bad it is. Not to dump on anyone in particular, but Keanu is surrounded by people speaking with accents yet his attempts are dismal, as are Winona’s. Lucy’s character is way oversexed. There’s a love plot device involving Dracula which seemed a little hackneyed. On the other hand, Gary Oldman and Anthony Hopkins are really great, but there’s too much happening for them to fix everything wrong here.

Now I started to wonder, is there something inherently not great about vampires? And just how accurate is this movie to the novel? Is the love story between Dracula and pants and/or Mina real? I had heard the novel was difficult to follow because it is told as a series of diary entries.

It took me over a month to get through this in audiobook but this book is great and it deserves its place as a classic. There is some sexiness but it is far from the principal element. There is a whole spooky ship scene which is rather great and doesn’t seem to be adapted often. Renfield is much more interesting in the novel.

The diary structure is not hard to follow at all, but it adds something wonderful to the atmosphere and the entire narrative structure. First, you get to see each character discover the horror in turn. When Jonathan first brushes off the supernatural, the foreboding is quite intense; later, each of the other characters have his same rationality at first and we get to see it stripped away one by one, a real horror moment. Second, it’s a useful device when in the middle of the novel, instead of narrating each character coming up to speed, we just have this exchange of diaries and then everyone is up to speed. Saves Bram from writing a lot of tedious dialogue about so-and-so telling so-and-so things we already know. Third, it allows him to play with our expectations. When we are getting diary entries from Mina in the final act and then we get one from Van Helsing instead, he creates doubt and tension about Mina right before the last entry. And finally, you get a bit of voyeuristic pleasure from the sensation that you’re rifling through a big pile of diaries containing something not quite believable. Indeed, the final words of the novel state this.

As an armchair language aficionado, I can also tell that Bram Stoker was one as well. He spends a great deal of time discussing the languages and accents of different characters. I don’t find his accents, particularly Van Helsing’s, especially believable, but I appreciate the attempt.

So that was great. I recommend reading or listening to the original. The films are pretty much trash. The Audible version with Alan Cumming and Tim Curry is very nice.

May 7, 2025 books movies horror