Let’s talk about ΩeΩ = 1. Here’s a great blackpenredpen video about calculating this value:
This got me thinking about J, because it has some interesting “adverbs” for doing these sorts of calculations. For starters, there is a built-in numerical derivative operator D., and the power adverb, which has a note that giving an infinite power computes the limit. I think of it as a fixed-point operator: it repeatedly feeds the output back in as an input until the input and output are equal, which is a fixed point. The power adverb turns out to be a hugely useful control construct in several ways—you can use it to describe various kinds of conditions as well as loops.
The first problem in J is figuring out how to express the function ΩeΩ - 1 = 0. A little help from 13 : produced this: 1 -~ ] * ^. As usual, you read J from right to left. Start with the fork ] * ^. ] is effectively id for functional programmers. ^ is exponentiation; the dyadic form, such as 2^3 calculates 23, for instance, and the monadic form ^x computes ex. The fork trick being that the result of applying the left and the right are fed in as the left and right arguments to the center verb, ] * ^ is exactly xex. 1 -~ exploits a little trick about J, that constants can occur in certain positions of a fork or hook and act as values instead of functions, and because ~ flips the arguments of a verb, 1 -~ ... at the start of a train is a great way to signify “… - 1” in a formula.
xex = 1 -~ ] * ^.
There turns out to be no need to determine the derivative, because we can use the D. adverb. In fact, there is an essay about Newton’s method on the J software site; I wound up having to change d. to D. for reasons I don’t understand. In the end, we wind up with this adverb definition: N =. 1 : '- u % u D. 1'. Applying this once looks like:
(1 -~ ] * ^) N ] 1
0.68394Applying it two more times looks like:
(1 -~ ] * ^) N ] 0.68394
0.577455
(1 -~ ] * ^) N ] 0.57745
0.56723But, we can just let the power adverb iterate it for us until we converge:
(1 -~ ] * ^) N^:_ ] 1
0.567143Not enough precision? We can always ask for more:
0j50 ": (1 -~ ] * ^) N^:_ ] 1
0.56714329040978384011140178699861280620098114013672In the end the whole program is these two lines:
N =. 1 : '- u % u D. 1
0j50 ": (1 -~ ] * ^) N^:_ ] 1
0.56714329040978384011140178699861280620098114013672Unbelievable.
Curious how many iterations it took? If you give a boxed value to ^:, it will give you all the intermediates. So we can use # to count them up and <_ to produce an infinity in a box and find out:
# 0j50 ": (1x -~ ] * ^) N^:(<_) ] 1x
317For six places, it only takes 6 tries:
# (1x -~ ] * ^) N^:(<_) ] 1x
6
(1x -~ ] * ^) N^:(<_) ] 1x
1 0.68394 0.577454 0.56723 0.567143 0.567143Update on D. and d.
There are a few differences between D. and d. in modern J. The version of Newton’s method on the wiki page actually does work for this definition of ΩeΩ = 1: <:@(] * ^). There is not a huge difference between this and 1 -~ ] * ^, just that in the original case we are performing an argument-swapped subtraction with 1. In the new case we are composing a decrement function. The derivative operator d. is able to symbolically differentiate this definition, probably by simply discarding increment and decrement in composition since such things would fall off in general according to the power rule:
<:@(] * ^) d. 1
^ + ] * ^
(] * ^) d. 1
^ + ] * ^This is just the fact that d/dx(xex - 1) = d/dx(xex) = xex + ex.
In short, the differences between D. and d. are:
d.performs symbolic differentiationD.performs numeric approximation of a derivativeD.has different rankD.can be used to produce partial derivatives as well
The following books have made an impression on me and my beliefs.
Biblical Literacy by Rabbi Joseph Telushkin. This book gives you a synopsis of the bible, and Telushkin draws your attention to the moments and ideas that are important in Judaism. In a similar vein, How to Read the Jewish Bible by Marc Brettler, also surveys the content of the Bible and helps to put it into a modern Jewish perspective. Robert Alter’s translation of the Hebrew bible The Five Books of Moses, as well as its continuation in several more volumes, is also a vital resource to me. Alter brings an incredible amount of knowledge of Hebrew and related languages to bear on his translation and has copious notes about the decisions to be made, as well as point out puns or inside jokes that would not be apparent in the English translation. Bringing this edition to Bible study is worth a lot. Etz Hayim is the Conservative movement’s official chumash; the translation is bland, but you get 2:1 notes to text, which is wonderful, including a lot of very interesting tangential material. I found the combination of these two books to be very effective for bible study.
In a different vein, Chuang Tzu, translated by Burton Watson. The inner chapters in particular are both powerful and beautiful, but there are lots of gems in the remaining chapters as well, if you can ignore some repetition. I find a lot of compelling ideas in Taoism, especially Chuang Tzu’s version of it, but there is beauty in Tao Te Ching, the Addiss/Lombardo translation being the one I have.
The Rise of Christianity by Rodney Stark. This book shines a light on the darkness between 70 CE and the moment Christianity became the religion of the Roman Empire. Where did it come from, how did it grow so quickly? This book gave me a deeper appreciation of what it is about Christianity that made it change the world, and gave me more compassion for my Christian friends. Similarly but for Islam, Introduction to Islam for Jews by Reuven Firestone clarified things greatly, by showing ways that Judaism and Islam are similar and different. I think this book would work with a non-Jewish audience.
The Other God by Yuri Stoyanov is a fascinating deep-dive into the history of dualist thought, from ancient Egypt, through Zoroastrianism to defunct and mostly lost religions like Manichaeism and Catharism. Similarly, Ancient Mystery Cults is a deep dive into Mithraism and the Eleusian mysteries, what is known anyway, and the book was an effective antidote to tropes like “Christianity is just Mithraism dressed-up.” The Indo-European ideas that reappeared in Mithraism were also the bedrock of Iranian religion, where the dualist religion of Zoroastrianism was born and many other religions underwent change: the topic of the book Religions of Iran by Richard Foltz. Although not so much about religion, Genghis Khan and the Making of the Modern World by Jack Weatherford illustrates how many modern ideas were born by the Mongols and the fruit of their conquest, in the neighborhood.
In the Dust of This Planet and volume 2 of the same series, Starry Speculative Corpse are excellent overviews of a lot of areas of inquiry. I hesitate to come down hard as a pessimist but negative theology conceptually is very appealing to me and my understanding of it comes from these two books, which are short and approachable. I also got a lot out of Emerson, especially “Self-Reliance” and “Circles.”
Two other philosophers made deep impressions on me: Henri Bergson, whose Creative Evolution kept me struggling in college but whose ideas about time, duration, evolution and intuition have stuck with me in some bastard form, and Jiddu Krishnamurti, who really only had one thing to say (paraphrased “you don’t need a teacher, which is handy because you can’t have one anyway”) but said it so audaciously and powerfully that he wound up attracting a following and having to say it repeatedly. Try Freedom from the Known. (He deserves bonus points for dissolving the cult he was brought up to lead.)
In more depth with Judaism, we must discuss Heschel, whose books are so powerful I haven’t finished any of them. God in Search of Man is the one I most want to finish, The Sabbath is one I actually have finished, I Asked for Wonder is to my recollection, quotes, but even his quotes are great.
Some dimensions of Jewish practice that are worth discussion: Everyday Holiness teaches the practice of mussar, which is a form of Jewish-religious self-help; an interesting topic for someday when I’m bored. Jewish Meditation is what it says on the tin, and Rabbi Kaplan also did translations of other important Kabbalistic works, especially the purported manual for making golems Sefer Yetzirah and proto-Zohar Bahir, both of which are only approachable thanks to his copious notes. Notes outweigh source matter by about 10:1 in each of those.
Some more comparative religion, The Sikhs by Patwant Singh is a nice overview introduction to the religion I read before Wikipedia. Ofudesaki is the holy text of a fairly obscure Japanese religion, Tenrikyo; I never made it further than about 10 pages.
More crap—I mean, stuff—in the Western esoteric traditions: The Secret Doctrine of the Rosicrucians explains their “philosophy” of vibrations and magnetism and whatever. Similarly, Alchemy Rediscovered and Restored gives you more of the wider story of “spiritual alchemy” rather than just the Rosicrucian “version” of it. Feel free to go full stupid with Practical Qabalah and The Mystical Qabalah, but be sure to pick up Aleister Crowley’s books at some point.
I’ve been investigating radio networking a bit lately for a side project I’m thinking about. This summarizes what I’ve learned.
Different radio networking technologies in the hobbyist space have a fundamental tradeoff between range, bandwidth and power usage. Increasing range or bandwidth tends to increase power usage. So one has to ask questions like, how much data do you want to exchange on your network, at what rate, how far apart are they, and will your devices be powered.
- WiFi is a high-bandwidth, high-power-use option with medium range
- LTE is a medium-bandwidth, high-power-use option with long range (if you have coverage) that is pay-to-use.
- Bluetooth Low Energy is medium-high bandwidth, medium-low energy usage with very short range.
You’ve probably run into these already at some point, so it will be a relief to learn that it’s fairly easy to build out hobby projects using these technologies. You may be surprised to learn that none of these are what Phillips Hue, Smart Things, Nest or any other home automation system uses. Why is that?
- Connecting to WiFi (as you know) is not fast, but if your device enters a sleep state, it will have to rejoin when it wakes up. If you have a 20-byte packet to send every minute, the vast majority of your energy use is going to be wasted on WiFi associating. Most residential IoT things would rather have longer battery life.
- Who wants to spend $5/month per LTE device in your house? And what’s going to happen if you exceed a few MB of data?
- BLE is explicitly a one-to-one network with low range. It’s intended to be a cable replacement. Everything you want to coordinate has to be within the same short distance of your controller.
Now, there are bleeding-edge solutions to many of these problems. There is a low-power WiFi standard and Bluetooth 5 Mesh. But, there are other technologies that might be interesting to a hobbyist. If we were making a chart of bandwidth, power, distance with the options above you’ll notice we don’t have any medium-long distance, low-bandwidth, low energy use options. Those are:
- ISM-band packet radio
- 802.15.4/Zigbee/Thread
- LoRa
These options are all medium-low bandwidth, medium to long range, low power use. ISM band packet radio is what you get using a BBC micro:bit radio. The lowest-level option is the packet radio, which is essentially the most straightforward spray of a packet of bits on a particular frequency. This is low power use, but also not very reliable. The distance will depend on the power level and the frequency, with lower frequencies having lower bandwidth but longer range. Your choices are going to be 433 MHz, 915 MHz or 2.4 GHz, at least in the US. The BBC micro:bit is essentially doing this using its Bluetooth radio when you use the packet radio.
Zigbee and Thread essentially add addressing and mesh networking on top of the packet radio. The range is tens of feet, so somewhat longer than BLE but not quite WiFi. The main difference between the two is that Zigbee has a complex suite of protocols for different types of appliance and it’s own simple addressing scheme, where Thread just lets you use IPv6 addresses to talk to devices and leaves it at that.
LoRa doubles down on distance, having a range measured in multiple miles or kilometers. However, the packet size is very small, around 50 bytes, and you really can’t send more than 1% of the time. LoRa devices listen only directly after a broadcast for a tiny amount of time. I’m not sure what this is for. Perhaps farm automation? It’s very cool technology though.
Anyway, this is basically what I’ve learned.
How would you chop a linked list in half? A trivial approach would be to just get the length of the list and then walk the list building a first-half copy by tracking the indexes until you get to half that length.
Thinking about it, I realized you could use the tortoise-and-hare approach from Floyd’s cycle detector to find the middle of the list: walk the list item-by-item and every-other-item at the same time; when the every-other-item list is exhausted, you’ve found the middle:
% bisect(+List, -First, -Last) is semidet.
% bisect(-List, -First, -Last) is multi.
% First and Last are equal-length sublists of List such that
% append(First, Last, List) is true and
% length(First, N), length(Last, N) is true.
bisect(L, First, Last) :- bisect(L, L, First, Last).
bisect(Xs, [], [], Xs).
bisect([X|Xs], [_,_|Ys], First, Last) :-
First = [X|NextFirst],
bisect(Xs, Ys, NextFirst, Last).The other trick here is passing the tail of the list to the recursive call, sort of like an inexpensive difference list trick to give us O(N) appending to a linked list.
I’m pleased to note that this just fails for free on non-even length lists. No stupid solutions where one list is longer than the other.
?- bisect([1,2,3,4], X, Y).
X = [1, 2],
Y = [3, 4].
?- bisect([1,2,3,4,5], X, Y).
false.
?- bisect([1,2,3,4,5,6], X, Y).
X = [1, 2, 3],
Y = [4, 5, 6].
?- bisect(Z, X, Y).
Z = X, X = Y, Y = [] ;
Z = [_24123766, _24123772],
X = [_24123766],
Y = [_24123772] ;
Z = [_24123766, _24123772, _24123784, _24123790],
X = [_24123766, _24123772],
Y = [_24123784, _24123790] ;
...Stack Overflow is going through some kind of asshole midlife crisis and this blog post is the corresponding spiritual Mustang GT.
Don’t be fatuous, Stack Overflow. Your culture is the way it is because of your rules. It’s not an accident that oh-my-stars we just peeked under the rock and discovered last week. Your rules created this monster.
What is the overriding principle of Stack Overflow? It’s that questions (and answers) have differing value. Some questions are just more useful than others. Same with answers. And the whole rest of the machine falls out of that concept. People who ask better questions are better. People who give better answers are better. The dialogue between a seeker and an answerer is garbage unless it leads to an answer. Commentary is almost worthless. Subjectivity is worthless! Only ask questions that can be answered! No, I will not recommend an off-site resource for you, noob!
Why are we assholes to people who ask questions that already have answers? Because your rules say to close questions that have already been asked.
I follow the Prolog tag. I can tell you about 1 out of every 20 or 50 questions is actually new. The rest are people wanting help with their homework. Most of their questions can be answered by reading the output from their terminal aloud to them. They need help. But the Stack Overflow goal, which is perfect answers to the best questions, is useless to them because they are essentially illiterate. If they were literate, they would read their console output, or their textbook, or their notes. Instead, they want you to do the work for them because Stack Overflow and the internet more generally has created a culture for them where there is simply no need for many programmers to read or think.
“All developers are free to participate”—but that participation is modulated by your rep. And with the vast majority of established engineers being white men, of course the bulk of women and minorities are not high-ranking Stack Overflow users. But the difference has nothing to do with institutional bias. It’s completely a function of the date you joined. Stack Overflow is an asshole to new users, period. Nobody knows or cares about your background, just that you’re a noob, what with your “hi” and your “thanks” and your not knowing Markdown and your asking a question that’s been asked a thousand times.
Stack Overflow seems determined to overthink their position and fuck it up somehow, first with Documentation and now this. Why did we all love Stack Overflow in the beginning? Because you got answers without all the commentary, the irrelevant massive signatures, the page 923 of this forum post, the “is this solution still good?” ten years after the fact. You beat the shit out of Experts Exchange, that’s what we loved about you. Who’s competing with you now? Nobody. So just shut up already. Making Stack Overflow a better partner in the evolution of my growth as a Jewish programmer is a meaningless and stupid goal for you. If I ask a question that already has a good answer, it’s a waste of everyone’s time to stop and answer it, and that’s really the only way to evolve from where you are now.
There’s a joke about contradictory requirements that goes something like, “I demand we build a new schoolhouse! I demand we use the bricks in the existing schoolhouse! I demand we keep the old school open while we build the new one!” Stack Overflow cannot be a welcoming place that caters to the wholeness of your being and still be the central resource for canonical answers to every question about programming. Those concepts cross each other exactly the same way “the bricks must be reused while the school stays open” crosses itself. You can’t be welcoming to new users and tell them to go fuck themselves for asking a duplicate question, and conversely you cannot be the home of the best comprehensive answers to the best questions while allowing your content to degrade into a billion answers of the same stupid questions because someone couldn’t be bothered to open their textbook before failing to implement their homework.
This is really disingenuous anyway, because you have, what, 50 developers against literally the rest of the world’s developers? Even if you had a great idea, you don’t have the manpower to police anything. Your police are, unsurprisingly, users with a lot of rep—in other words, the people most invested in the way the system currently works.
In short: you have no concrete plan, you have no resources to implement a concrete plan if you had one, and you have no actual incentive to do it. So this is nothing. Posturing-as-a-product.
The architect at my work recently handed a prototype he build to me and I was instructed to maintain it. In so doing, I have found a nice little parable here about how things can go wrong and how it can spiral out of control. Also, let’s spit on Hibernate.
At the outset, he chose to build a REST server with Hibernate. He went with RESTEasy, which antedates Jersey (JAX-RS/JSR-339) by a few years. The first problem in our story happens because he is using an old version of RESTEasy and an older version of Hibernate. Namely, the JSON writing portion (Jackson) gets confused by the Hibernate proxies to the model objects he’s retrieving and he proceeds to annotate most of the Hibernate mapping with lazy="false".
We have seen in the past that this is a common source of pain because Hibernate actually has two settings here where it should have one. The setting lazy="false" tells Hibernate, you must immediately retrieve these properties. But, another setting (such as fetch="subselect") actually tells Hibernate how to do this, and without it, you get a nice combinatorial explosion of queries as Hibernate must iterate row by row running queries that lead to it iterating again running queries… a single web request can easily balloon into several hundred thousand queries. Also, there were no indexes, so these queries were running full table scans for essentially every foreign-key based retrieval.
The immediate problem of JSON not being generated is solved, but the “root cause” problem of many other issues is born. Now many of the requests are slow. The next stop is apparently Angular. The best way to avoid expensive work is to not do it, followed by caching the results. Now he introduces a significant caching layer in Angular, in the browser. He does this before implementing pagination.
New problem. Rendering time appears bloated. (I think this is conflated with fetch time.) Solution: angular link-time string concatenation rather than using ng-repeat which generates watches recursively. Rendering time improves. Code is much larger and more brittle.
New problem. Cached data is out-of-date. Solution: create a web socket and tie into Hibernate’s interceptor system to broadcast notifications whenever entities are persisted. Bunch of new code for handling notifications and updating the relevant caches.
The Hibernate-friendly solution to these problems is as follows:
- It’s possible to annotate the model classes to hide Hibernate proxy properties, so that Hibernate proxies can be converted to JSON, even with very old versions of Jackson and RESTeasy
- Remove the
lazy="false"or addfetch="subselect", depending on whether it is actually a performance improvement to fetch immediately - Replace generic fetches of objects with JQL queries using
INNER JOIN FETCHthat obtain exactly the entities needed at the time they are needed - Add foreign key indexes to the database, and others as-needed
- Delete the cache layer, since it may be hiding performance issues
- Delete the web socket notification layer, or make it induce a reload of the page rather than tinkering with cached values that may not even be displayed
However, I have a more holistic approach I would like to recommend, because the current scheme looks something like this:
stupid model with lots of MVC
This is stupid. You’re doing a lot of changing the shape of data, but it’s you in the front-end, it’s you in the back-end and it’s you in the database. So just admit that the person wearing the front-end hat is the same person wearing the back-end hat and the person writing the database, and let each of these components do the part of their job they are good at:
less stupid model with only one MVC
There. Now your back-end can be honest and have the SQL, yes SQL, it needs to build exactly the views that your front-end provides. Hibernate doesn’t need to be involved here because it isn’t buying you anything to take your fat objects out of the database at great cost, transmit them in all their bulk to the front-end, have them taking up space in the user’s browser (which is probably the worst at managing the memory) only to have your Angular front-end iterate through them and render four properties in a table view. You’re spending in the database, fetching stuff you don’t need, spending in the back end to convert it from one format to another, and then doing the same conversion again in the front-end. Caching is not the solution here! Doing less work is the solution! Build a holistic solution rather than three incomplete pieces of a solution!
So think holistically. Use mybatis and actually learn some SQL. Make the database do the work of changing the shape of the data—that’s what it’s for! Make the back-end do things to the data: this is what brings action and processing to your data. And then, make the front-end just show the damn data! That’s all it needs to do: display the information and collect the user’s desires for processing and pass it back. THIS IS AN MVC SYSTEM ALREADY—it doesn’t help to make the model MVC, the view MVC and the controller MVC!
In conclusion, Hibernate is garbage.
Addendum
I should make it clear I have the utmost respect for the architect. The high-level ideas in this program are great. Their execution is essentially befitting of a prototype. I don’t think I would have come up with as powerful, general or interesting of a design if left to my own devices.