⅔ non-programming. Zola 2026-02-19T05:39:06+00:00 https://briankung.dev/atom.xml 2026-02-19T05:39:06+00:00 2026-02-19T05:39:06+00:00 Unknown https://briankung.dev/2026/02/19/on-ai-assisted-coding/ <p>A lot has happened in software engineering since my last post. I've had a draft post called "LLMs are crazy good" for a while now, but suffice to say that LLMs have gotten even better at coding since then, to the degree that they are now essentially my preferred way to edit the text files that we call code.</p> <p>Also, it feels silly to have to write this, but every word on this blog has been and always will be written 100% by a human, except for code. I would say "artisanally hand-crafted," but some days it's more like slapped together with a stapler and chucked out the window. This is one of those days, to be honest.</p> <h2 id="llms-are-an-abstraction-layer-comparable-to-compilers">LLMs are an abstraction layer comparable to compilers</h2> <p>The basic idea is that LLMs allow us to move up another abstraction level, similar to how compilers did in the 1950s (thank you, Admiral Grace Hopper). Were a lot of people concerned about the output of compilers back then? They sure were. Are they now? Only compiler engineers. Are a lot of people concerned about the output of LLMs today? They sure are. Will they be concerned about the output of LLMs tomorrow? I honestly don't think so.</p> <p>I think as we learn to use LLMs for code generation, we will find out more ways to make the output deterministic <em>in the ways that we care about</em>. Do I care whether an API is written in Ruby or Rust? Do I care whether a website was written with htmx or erb? No, I don't.</p> <p>Someone wrote a blog post that probably thought a lot more about the analogy which you should definitely read: <a rel="external" href="https://alperenkeles.com/posts/llms-could-be-but-shouldnt-be-compilers/">https://alperenkeles.com/posts/llms-could-be-but-shouldnt-be-compilers/</a> My one comment on the contents of the post would be that 1) compilers aren't entirely deterministic, either, and 2) we can and <em>will</em> find ways to increase the reliability of LLM output.</p> <p>Side story: My dad told me a story about picking flooring for a home remodeling project. He went to the flooring store with my mom and they were agonizing over their choices. The sales rep gave them a few options and then, after witnessing the agony, shrugged, and said "it's just money" and laughed. In our case, the sales rep is Anthropic and they're saying "it's just tokens."</p> <p>The point is, you can now waste tokens to talk through the specification with your LLM agent. And speaking of specifications...</p> <h2 id="the-specification-is-the-application-now">The specification <em>is</em> the application now</h2> <p>These days, the specification is the application, now, or at least more so than before. LLMs do much better when there's some sort of deterministic output they can test their code against. For example, JustHTML passes 100% of the html5lib test suite and was entirely vibecoded: <a rel="external" href="https://friendlybit.com/python/writing-justhtml-with-coding-agents/">https://friendlybit.com/python/writing-justhtml-with-coding-agents/</a> And they were able to do it because <em>there is a test suite in the first place.</em> Meanwhile, Anthropic had Claude Opus 4.6 build a working C compiler that can compile the Linux kernel. The trick was to have a reference compiler and a great harness for writing a clean room implementation.</p> <p>These, on the spectrum of "formally specified" on one end to "bro, I have an idea for an app that's like Strava plus Hinge" are definitely much closer to "formally specified." There's a clear feedback loop that tells the LLM objectively whether it's headed in the right direction, and a test suite that lets it iterate toward the goal. There's actually a library that is just a list of specifications of the behavior. If you can find that library for me, I would appreciate it.</p> <p>I feel like <a rel="external" href="https://cucumber.io/">Cucumber</a> needs to make a comeback, but this time powered by LLMs trying to playtest your website with Playwright:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="gherkin"><span class="giallo-l"><span style="color: #8839EF;">When </span><span>logged in as a User I should be able to personally send Anthropic&#39;s stock price to the moon 🚀</span></span></code></pre> <p>So how does this affect software development? We're going to be much more in the business of specifying behaviors rather than writing fiddly text keywords to match syntactical rules. I think it's inevitable. It's not just a matter of cost and efficiency - it's just a better way to code. But it is all of those things as well.</p> <h2 id="llms-are-fpgas-and-runtimes-are-asics">LLMs are FPGAs and runtimes are ASICs</h2> <p>FPGAs, or Field Programmable Gate Arrays, are hardware based logic boards that can be programmed to represent different logical circuits. When TSMC tapes out a chip, the design is always going to be a certain way. FPGAs let you program the physical design to your specification and it will behave as if it was a hardware circuit board design, albeit slower and less capable than the real thing.</p> <p>ASICs, on the other hand, are Application Specific Integrated Circuits, and they are fully specified circuit boards that achieve a single purpose. These are often desirable because they're cheaper to produce and perform much quicker than FPGAs or more general purpose designs.</p> <p>In our analogy, LLMs are FPGAs. You can load an LLM up and tell it to pretend to be a command line terminal and it will respond as if you were <code>ssh</code>'ing into a remote computer somewhere. They are the ultimate general purpose, if somewhat chaotic, computing tools. <em>However</em>. They can be used to encode processes into code. And code is far less chaotic than an LLM is, and that is a benefit when you want to <em>use</em> a tool instead of learn how to use the latest model. It's also typically far less resource intensive than waiting for inference from an LLM. If you're using the same runtime and the same code, you should more often than not get the same result.</p> <h2 id="coding-feels-dead">Coding feels dead</h2> <p>I follow coding subreddits like Rust and Ruby and I have to say - it all feels dead now. There are fewer high quality posts, and the excitement is all in LLM submissions that, for better or for worse, have a much higher bar to meet before readers consider them remarkable. The magic feels simultaneously over and underwhelming. Software engineering used to feel like a delicate ballet of computers, orchestrated by a careful choreographers.</p> <p>Now you can get world class software engineering knowledge at the touch of a button, if you ask the right questions. It's great, but it's also a kind of <a rel="external" href="https://simonwillison.net/2026/Feb/15/deep-blue/">Deep Blue</a>, a melancholy that Kasparov and later Lee Sedol must have faced when they were the first humans to be thoroughly defeated by computers in the games of Chess and Go, respectively.</p> <p>I think that, despite the melancholy, we will adapt like the board gaming communities have. The game is still about people, but we will use AI to enhance our own training and our own capabilities. The melancholy is that we've built something that so far outpaces our capacity that we are, in a way, puppets of the machines rather than the operators.</p> <p>In truth, the entire internet feels dead now, but this isn't a <a rel="external" href="https://en.wikipedia.org/wiki/Dead_Internet_theory">new theory</a>. It's just that the internet is now aggressively dead, like a <a rel="external" href="https://www.youtube.com/watch?v=mWEhfF27O0c">28 Days Later</a> kind of dead rather than an Abraham Lincoln sort of dead.</p> <h2 id="on-trusting-trust">On trusting trust</h2> <p>Another compiler analogy, but the seminal paper <a rel="external" href="https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf">"Reflections on Trusting Trust"</a> explores how a backdoor in a compiler could lead to almost completely undetectable security vulnerabilities.</p> <p>In LLMS, we're already seeing OpenAI talk about ads, but the problem of trusting trust is actually even worse than the problem in compilers, since compilers are actually somewhat transparent compared to LLM models. So much of the model training process is obfuscated, from the data sources (like Facebook <a rel="external" href="https://arstechnica.com/tech-policy/2025/07/meta-pirated-and-seeded-porn-for-years-to-train-ai-lawsuit-says/">infamously claiming</a> that its seeding 81.7TB of porn was for "personal use") to the training harnesses, to the processes, and at the end you get a massive binary blob of <em>statistics</em> somehow representing the sum knowledge of humanity. A hypercube of billions of parameters for the pachinko ball of your piddly prompt to bounce through until you get an answer you're satisfied with.</p> <p>No one but the most well funded AI labs can introspect these things. And the actors on the web are quickly adapting to poison the wells the LLMs came from, whether from careless neglect and AI slop, or outright malice and pushing narratives to their own purposes. And finally, capital must always have its say, and so we have the ads injected into every prompt. I wouldn't be surprised if it was eventually a service to have ads injected into the training data itself, to permanently bias LLMs, though I have my doubts as to the economics of that business model.</p> <h2 id="ethics">Ethics</h2> <p>If you're coming from a Western perspective of intellectual property (as opposed to, say, a <a rel="external" href="https://www.bunniestudios.com/blog/the-12-gongkai-phone/">gongkai</a> perspective), then AI is inherently unethical. It's based on intellectual property laundering that washes away ownership through statistics. Image generation models all know what they're <em>not</em> supposed to generate when you ask them for Avengers, Disney characters, Ghibli characters, or Pokemon. They absolutely can and will generate faithful recreations of all of the above, however.</p> <p>LLMs are also yet another source of emissions that contribute to climate change, and use precious resources. In the most egregious examples, they directly compromise vulnerable populations with their emissions when the electricity grid can't meet the AI datacenter demands.</p> <p>And yet, this technology won't go away. We can spurn it, we can denounce it, but we must in some ways engage with it.</p> <p>The ethics of AI are not that much different than deciding to take an airplane somewhere, drive your car, or remodel your house. Everything we do, especially what we do out of convenience, creates waste. All waste has externalities and all externalities are paid for by the common folk, ultimately.</p> <p>The problem is upstream of "do I use LLMs today?" and it's upstream of "do I drive a car to work today?" In my opinion, we need to do our individual parts to reduce waste and the effect on the global good that is the health of our globe, but we must also balance that against the need to hold society and particularly corporations accountable for their share of the tragedy of the commons.</p> <p>We should have technology that we can depend on as a public good, efficient infrastructure that sips resources, equitable access to the rewards that come from progress. That, unfortunately, is something that no prompt can solve. Only we can hold other humans accountable, and only we can share the fruits of our labors with others.</p> 2025-04-18T03:17:15+00:00 2025-04-18T03:17:15+00:00 Unknown https://briankung.dev/2025/04/18/only-software-engineering-can-save-us-from-vibe-coding/ <p>I saw a "stop learning to code, vibe coding is here now" post and I had to write a response: While it's true that AI has commoditized the act of stringing characters together into code, it hasn't commoditized software engineering. In fact, it's made software engineering more important and more accessible.</p> <p>The difference between coding and software engineering is that software engineers know that every line of code is a liability. Consider this quote (purportedly) by Bill Gates:</p> <blockquote> <p>“Measuring programming progress by lines of code is like measuring aircraft building progress by weight.”</p> </blockquote> <p>Code is a liability for the same reason that excess weight is a liability for aircraft. Code needs to be maintained, and in order to be maintained, it needs to be understood. It is also a liability because, like any sharp tool, it can cause damage when executing in the wrong context. Code inherently runs against its own purposes because code is intended to solve a problem, but it creates its own problems by dint of existing.</p> <p>So I think of the maintainable use, modification, and execution of code as software engineering. What does that mean?</p> <ul> <li> <p>Fast, meaningful tests</p> </li> <li> <p>Observability</p> </li> <li> <p>Documentation</p> </li> <li> <p>Reproducibility</p> </li> </ul> <p>Can AI do these things? Well, it can certainly help. Will it if you don't steer it towards them? No, it won't. But, crucially, if you do have these elements, it is a flywheel that safely accelerates any modification to your system, whether it's coming from a user or from an LLM.</p> <p>So please, if you only want to learn vibe coding, learn software engineering next. I promise the vibes will be immaculate when your test suite is green.</p> 2024-06-11T22:56:51+00:00 2024-06-11T22:56:51+00:00 Unknown https://briankung.dev/2024/06/11/tiny-overly-excited-explainer-about-vector-search/ <p>Random thing that I got excited about this morning because I wanted to add really nice search to my personal journal: vector embeddings.</p> <p>Imagine a 3D Cartesian space, and in it there are words at various (x, y, z) points. Similar words are placed in similar locations. In this space, "king" and "queen" would be close to each other, whereas "beggar" would be far away. This is essentially what a "vector embedding" is, except that vector embeddings typically have a much higher dimensionality than 3D space.</p> <p>For an example of a dimension, take the relationship between "hot" and "cold." You might track that as a dimension to judge a word by. How about social vs antisocial? Or fast vs slow? If you rated each word on hundreds of such dimensions, you'd have a single vector in a high dimensional space:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="javascript"><span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">// vector embedding for the word &quot;ice&quot;</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>[</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"> // hotness</span></span> <span class="giallo-l"><span style="color: #FE640B;"> 0.0</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"> // coldness</span></span> <span class="giallo-l"><span style="color: #FE640B;"> 1.0</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"> // social</span></span> <span class="giallo-l"><span style="color: #FE640B;"> 0</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"> // antisocial</span></span> <span class="giallo-l"><span style="color: #FE640B;"> 0.3</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"> // fast</span></span> <span class="giallo-l"><span style="color: #FE640B;"> 0.1</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"> // slow</span></span> <span class="giallo-l"><span style="color: #FE640B;"> 0.5</span></span> <span class="giallo-l"><span>]</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">// Without comments:</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>[</span><span style="color: #FE640B;">0.0</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;"> 1.0</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;"> 0</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;"> 0.3</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;"> 0.1</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;"> 0.5</span><span>]</span></span></code></pre> <p>That vector space is technically a 6-dimensional hypercube, but it's easier to visualize vector embeddings as being 3-dimensional.</p> <p>The placement of words within the hypercube is done via arcane magic. In fact, the dimensions don't really correlate to anything we could solidly name, unlike my example, and the "words" are actually fragments of words, text, or even bytes - they're certainly not complete words. That's why these neural networks understand that "quick" and "quickly" are similar.</p> <p>Regardless, the end result is that you can take some text, run it through a neural network, and get the position of that text within the hypercube. Then you can see if there's anything nearby that has the same semantic meaning. That way, when you search "monarch man," the embeddings will return "king" instead of failing like traditional full-text search, which would require you to search "k," "ki," "kin," or "king."</p> <p>Creating the neural networks takes a lot of computing power, but once they're done, you can download them for your own use in order to 1) convert words or phrases into positions in the hypercube, and 2) search the space for nearby entries instead of searching text for exact matches.</p> <p>Anyway, it'd be a huge improvement to search on my personal daily journal, so I would like to get it working. Someday, maybe.</p> 2024-02-07T17:04:24+00:00 2024-02-07T17:04:24+00:00 Unknown https://briankung.dev/2024/02/07/tests-are-an-observability-concern/ <p>The first thing I do when I encounter a strange code base is run the test suite. I do this because I can get an idea of:</p> <ol> <li> <p>the exact use cases that the developers envisioned</p> </li> <li> <p>the actual usability of the APIs</p> </li> <li> <p>the overall health (read: test coverage) of the project</p> </li> <li> <p>and get an idea of how development is done</p> </li> </ol> <p>I've heard a lot about the benefits of test suites, even in the context of articles that are against TDD (test-driven development). For instance:</p> <ol> <li> <p>They're for regression testing</p> </li> <li> <p>They inform the design of the software</p> </li> <li> <p>They are a way of organizing the work into small chunks</p> </li> </ol> <p>And conversely, the negatives:</p> <ol> <li> <p>They are code, and code is a liability</p> </li> <li> <p>They are slow to write and maintain or refactor</p> </li> <li> <p>You need to know what you are doing in order to write a test</p> </li> </ol> <p>I'm not here to argue with any of these points, and I think different points are valid for different people and different situations. What I am interested in exploring is the idea that tests are a part of your code's observability stack:</p> <blockquote> <p>Observability is a system property that defines the degree to which the system can generate actionable insights. It allows users to understand a system’s state from these external outputs and take (corrective) action.</p> <p><a rel="external" href="https://glossary.cncf.io/observability/">Observability | Cloud Native Glossary</a></p> </blockquote> <p>Tests are simple red/green line-item indicators for the functionality of your system. In this case, the "system" is your code and the "user" is the developer. When a test fails, it is an "actionable insight" allowing the user to understand the code's state and take action (you could also argue that testing the public interfaces of code makes it an "external" output).</p> <p>A system that simply performs its function with no affordances for observability is a black box that doesn't lend itself to change. Without indicators of what it's doing or how it functions, it's nearly impossible to have any confidence in its performance or methodology.</p> <p>Imagine running <code>./bin/server</code> and being greeted with a blank screen. Is it running? What port is it listening on? What does it do? What is its API like?</p> <p>A code base without a test suite is like a server with no logs. As someone tasked with the maintenance of the code base, developers are higher order "operators" of sorts, where the output of the operation is a functioning program.</p> <p>Ultimately, this post is less about the importance of tests and more about the importance of observability. Being able to see into our systems is crucial in order to maintain them, and tests are a front-line mechanism for deriving <em>actionable insights</em> from the state of your code.</p> <blockquote> <p>To me, <em>legacy code</em> is simply code without tests. […] Code without tests is bad code. It doesn't matter how well written it is; it doesn't matter how pretty or object-oriented or well-encapsulated it is. With tests, we can change the behavior of our code quickly and verifiably. Without them, we really don't know if our code is getting better or worse.</p> <p>Michael Feathers, Working Effectively with Legacy Code</p> </blockquote> <p>Observability at any level may not be a priority for a two-person startup, but it is guaranteed to become a critical concern the more successful a project becomes. The more observable your system is, the easier it is to fix or change.</p> <p>I'll close out with an analogy that I've been toying with: a test suite is basically the nervous system of code. Making a change that triggers a regression in the unit test is like stretching in the morning and rediscovering that yesterday was leg day. Your nerves tell you swiftly where the affected area is, and you can move to correct the error. Without your nerves to tell you where and what is happening, you won't be able to react appropriately to the state of your health. The same goes with a good unit test. Without tests, your code is numb to changes and prone to stumbling.</p> 2024-01-26T19:22:59+00:00 2024-01-26T19:22:59+00:00 Unknown https://briankung.dev/2024/01/26/implementing-the-raft-consensus-algorithm-with-david-beazley/ <p>I survived David Beazley's weeklong course on the Raft consensus algorithm that powers technologies like Kubernetes, MongoDB, and Neo4j.</p> <p><a rel="external" href="https://raft.github.io/"><img src="/assets/images/solo.svg" alt="A cute drawing of three logs tied together into one &quot;Raft&quot; with a smiling face" /></a></p> <p><em>Image from <a rel="external" href="https://raft.github.io/">https://raft.github.io/</a></em></p> <p>The Raft Consensus Algorithm is a way for a gaggle of computers to agree on a sequence of events, or a "log" of events. Raft is useful for things like databases - once your data spans multiple computers, you will have consistency issues. Imagine a group of friends who want to go on vacation. They each have an itinerary in mind, but if each one has a different idea for what to do on a given day, there will be chaos. In this scenario, each friend is a computer and the itinerary is the sequence of events that they're trying to keep synchronized.</p> <p>In broad swaths, Raft is about keeping those logs synchronized through a democratic election process. Initially, every server starts out as a peer and then after a time, one will decide to solicit votes from their peers to become a leader. If a majority vote for that peer, then that server will become a leader and only then will the group as a whole become operational.</p> <p>Going back to the vacation itinerary analogy, everyone starts out on the same page until one of the friends takes the initiative to start planning. Once everyone votes a leader in, people can start to add items to the itinerary by telling the leader, who then replicates those items to its followers.</p> <p>Having a majority of servers operational and acknowledging the leader is crucial to this algorithm due to the fact that servers may go down, or the network may break down. This algorithm is intended to maintain availability in the face of failure. So only data that has been "committed" by a majority of the servers is acknowledged by the leader. If less than a majority of peers has been able to write an "itinerary item," then that item is considered un-committed and can be overwritten by the next item that comes in.</p> <p>There are countless more details that go into the algorithm, like timeouts triggering new elections in case a leader goes down, log replication rules for log entries that conflict with each other in case there are rogue leaders, and carefully tracking term and item numbers to validate that a leader is, in fact, legitimate, as well as concerns about how to interface with external applications.</p> <p>The class, which Beazley calls "Rafting Trip" (<a rel="external" href="https://dabeaz.com/raft.html">https://dabeaz.com/raft.html</a>), consisted of 30-60 minute long discussion sessions followed by up to 2 hours of coding, with a break in the middle around 1pm central time, and then another afternoon session. All in all, days went from 9:30am to 5:30pm. I was grateful to have my wife (who is also juggling work!) and my dad watching the kids while I focused on the class; otherwise this would have been impossible.</p> <p>It was a humbling experience. For comparison, Beazley likened the complexity to writing an operating system by the last day (today), or even harder, and he would know since he taught the operating systems course at a university level.</p> <p>Beazley's instruction style was informal, but also probing. He treated us all as colleagues simply exploring an interesting and devilishly complex problem - he even coded a Raft implementation himself alongside us. It was really refreshing to be able to focus on a problem in depth with some like-minded programmers instead of playing kick-the-can with technical debt because of business needs.</p> <p>I decided to attempt to use Rust to implement the challenge solutions, which was certainly <em>enlightening</em>, if exceedingly painful (in a good way, I promise).</p> <p>Every time I choose Rust for a challenging programming problem it does its best to make me regret it, and then I come out with a stronger conviction to use it again next time. Rust's appeal is mostly masochistic; convince me otherwise.</p> <p>Since the algorithm has to do with multiple systems in play, I not only sharpened up on my multithreaded programming, but also on fundamentals of networking. Unfortunately, I didn't get far enough along to have my multithreaded programs talk to each other, but I was able to get Udp packets sent from program to program and finally exercise my brain on the differences between sharing state within a thread (reference counted: <code>Rc</code>), across threads (atomically reference counted: <code>Arc</code>), and various mechanisms for sharing or mutating data like <code>RefCell</code>, <code>Mutex</code>, and <code>RwLock</code>, which are for interior mutability, mutually exclusive threadsafe access, and concurrent threadsafe access, respectively.</p> <p>My strategy when implementing Raft's election process was to focus on a <code>Channel</code> trait that I could mock out in order to mimic messages being sent. The thought was to make the system testable by faking interactions between servers, which I called <code>RaftBuddy</code>s because my brain was melting at the time and it amused me. Later, I could implement the <code>Channel</code> trait to actually talk to the network. Alas, everyone has a plan until the plan punches you in the face with a faulty Raft implementation.</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">// My poor vestigial Channel implementationuse crate::raft_message::RaftMessage;#[derive(PartialEq, Eq, Clone, Default)]pub struct RaftChannel { pub queue: Vec&lt;RaftMessage&gt;,}pub trait Channel { fn push(&amp;mut self, message: RaftMessage); fn pop(&amp;mut self) -&gt; Option&lt;RaftMessage&gt;; fn all_messages(&amp;mut self) -&gt; Vec&lt;RaftMessage&gt;;}impl Channel for RaftChannel { fn push(&amp;mut self, message: RaftMessage) { todo!() } fn pop(&amp;mut self) -&gt; Option&lt;RaftMessage&gt; { todo!() } fn all_messages(&amp;mut self) -&gt; Vec&lt;RaftMessage&gt; { todo!() }}#[derive(PartialEq, Eq, Clone, Default)]struct TestRaftChannel { queue: Vec&lt;RaftMessage&gt;,}impl Channel for TestRaftChannel { fn push(&amp;mut self, message: RaftMessage) { self.queue.push(message); } fn pop(&amp;mut self) -&gt; Option&lt;RaftMessage&gt; { self.queue.pop() } fn all_messages(&amp;mut self) -&gt; Vec&lt;RaftMessage&gt; { self.queue.clone() }}</span></span></code></pre> <p><em>(Do I actually think <code>all_messages</code> is a good idea as a public interface? No, but it made testing easier and I was more focused on desperately shoving the wreckage of my Raft implementation around, like a college student trying to make his dorm room presentable before his parents visit.)</em></p> <p>I got as far as leader elections requiring a majority before the end of the week. A lot of it was managing my energy levels. Oftentimes, I would just lie on the floor and cry think about the problem. I thought this was going to be difficult coming in, and I still underestimated how difficult it was.</p> <p>If I were to go back in time, I would probably make a point to do the projects in Ruby. It was hard enough to complete the projects, but learning the proper data structures to do multithreaded Rust as well definitely distracted from the Raft algorithm itself. However, now that I've stumbled through an initial attempt, I am pretty happy with what I learned and if I were to <em>redo</em> the course, I'd gladly do it in Rust again.</p> <p>On that point, it did seem like Rust's type safety, when applied to multithreaded code, did end up having some advantages over the dynamically typed Python code that many of my classmates were using. While we all wrote multithreaded code, Rust's enforcement of the appropriate data types gave me a lot more trouble upfront, whereas other participants in the class seemed to have more issues debugging issues with shared mutable state. So I'm not sure a Ruby rewrite would be much easier - rather, it would just present a different set of problems.</p> <p>That's why when David asked what we would do differently, my only real feedback was that I would have started a month ago. The algorithm is so heinously complex that I don't think any particular change would have been any easier. I'm certain I would have faced just as many issues in the end.</p> <p>This course and the algorithm were both ridiculously difficult, humbling, and extremely painful. I would highly recommend taking it if you ever have a chance, and now I'm eyeing David Beazley's other courses as well.</p> <p>Links:</p> <ul> <li> <p><a rel="external" href="https://raft.github.io/">https://raft.github.io/</a></p> </li> <li> <p><a rel="external" href="https://github.com/briankung/rafting_trip">https://github.com/briankung/rafting_trip</a></p> </li> <li> <p><a rel="external" href="https://dabeaz.com/raft.html">https://dabeaz.com/raft.html</a></p> </li> <li> <p><a rel="external" href="http://dabeaz.com/courses.html">http://dabeaz.com/courses.html</a></p> </li> </ul> 2023-07-27T09:11:30+00:00 2023-07-27T09:11:30+00:00 Unknown https://briankung.dev/2023/07/27/staff-engineer-by-will-larson/ <p><a rel="external" href="https://staffeng.com/book">Staff Engineer by Will Larson</a> talks about what happens in your career should you decide to progress past Senior Engineer. For context, software engineers typically progress through a series of well known levels: junior, mid (or just "engineer"), and senior.</p> <p>All of these levels are characterized by increasing autonomy and impact:</p> <table><thead><tr><th>Title</th><th>Autonomy</th><th>Impact</th></tr></thead><tbody> <tr><td>Junior Engineer</td><td>Needs direction</td><td>Needs assistance completing tasks</td></tr> <tr><td>Mid-level Engineer</td><td>Self-directed for day to day tasks</td><td>Independently completes tasks</td></tr> <tr><td>Senior Engineer</td><td>Self-directed, assists with team direction</td><td>Independently completes tasks with high level of quality</td></tr> </tbody></table> <p>Senior Engineer is what's called a "terminal level," meaning you could spend the rest of your career as a senior engineer. Basically it's an equilibrium point; another way to think of it is that you won't be let go for deciding *not* to go for a promotion, whereas for more junior positions that is generally not true (though I feel like Engineers get by just fine in my experience).</p> <p>Staff plus levels, meaning Staff / Principal / Fellow / Distinguished Engineers are engineers who are still individual contributors, but nonetheless take on leadership roles. So the table begins to look like this:</p> <table><thead><tr><th>Title</th><th>Autonomy</th><th>Impact</th></tr></thead><tbody> <tr><td>Staff Engineer</td><td>Influences a team or division</td><td>Affects the output of team members</td></tr> <tr><td>Principal Engineer</td><td>Influences the organization</td><td>Affects the output of teams</td></tr> <tr><td>Distinguished Engineer</td><td>Influences the industry</td><td>Affects the industry</td></tr> </tbody></table> <p>Each subsequent level comes with increasing understanding of the business domain and increasing organizational trust in their abilities to do what's best for the business.</p> <p>Q: So if it's a leadership role, what's the difference between Staff+ and going into management?</p> <p>A: The difference is in the reporting structure. As a staff+ engineer, you're not in the chain of command and it's not typically your job to do performance reviews, 1:1's, or firing, yet you have the mandate to operate outside of your immediate chain of command in order to maximize impact on the organization. This lets you "go to ground" among your fellow engineers and get their unfiltered opinions on the state of engineering in the organization without fear of managerial retribution. The difference is that staff+ roles are about soft power leadership, whereas managers are endowed with institutional power, which changes the relationship with other individual contributors.</p> <p>Also, sometimes being a force multiplier for a team can mean improving developer experience, platform engineering improvements, changing the onboarding or hiring processes, introducing a new technology, or promoting a psychologically safe working environment.</p> <p>Larson's book covers four archetypes of Staff+ engineers that stuck with me:</p> <ol> <li> <p>The Team Lead - the most common Staff+ archetype, the team lead is responsible for leading a team on an initiative or a domain.</p> </li> <li> <p>The Solver - the solver finds hot spots or particularly thorny problems and fixes them, often bouncing from problem to problem. Interestingly Larson notes that this archetype shows up in orgs with a "weak team concept."</p> </li> <li> <p>The Architect - the architect is responsible for architecting new solutions. This sounds simple, but good architecture accounts for brownfield projects as well, meaning incremental architectures like the <a rel="external" href="https://learn.microsoft.com/en-us/azure/architecture/patterns/strangler-fig">strangler fig pattern</a> come into play.</p> </li> <li> <p>The Right Hand - the right hand is the least intuitive to me. To my understanding, it's a form of very deep delegation to engineering leaders who act as representatives of an individual or an office. They force multiply leadership instead of force multiplying individual contributors like many of the other archetypes do.</p> </li> </ol> <p>After discussing the archetypes, the book then goes into tips for how to operate as a Staff+ engineer, then how to get the position, whether that's at your current company or otherwise, then filled the last half of the book with all the source interviews that the author did with Staff+ engineers.</p> <p>Overall, the book covered a lot of ground - almost too much, because it will often allude to linked articles that aren't summarized at all. I do think it would have benefited from a professional editor. However, while the book feels a bit like a long blog post, it's still very useful and does well in conveying the overall idea of the various Staff+ positions.</p> <p>I found the interviews at the end possibly more useful than the first half of the book in terms of getting an idea of what real staff+ engineers do - having cohesive stories makes it easier to remember them, even though I had read excerpts from the interviews in the first half of the book.</p> <p>I'm very curious to compare this to Tanya Reilly's book, <a rel="external" href="https://www.oreilly.com/library/view/the-staff-engineers/9781098118723/">The Staff Engineer's Path</a>, which is up next.</p> 2023-07-16T06:02:27+00:00 2023-07-16T06:02:27+00:00 Unknown https://briankung.dev/2023/07/16/rust-notes/ <p>I thought I'd jot down some notes about using Rust that wouldn't have made it as their own blog posts:</p> <ol> <li><a rel="external" href="https://briankung.dev/2023/07/16/rust-notes/#you-can-t-use-anonymous-functions-for-control-flow">You can't use anonymous functions for control flow</a></li> <li><a rel="external" href="https://briankung.dev/2023/07/16/rust-notes/#the-build-times">The build times</a></li> <li><a rel="external" href="https://briankung.dev/2023/07/16/rust-notes/#traits-traits-traits-and-more-traits">Traits, traits, traits, and more traits</a></li> <li><a rel="external" href="https://briankung.dev/2023/07/16/rust-notes/#keeping-up-with-rust-news">Keeping up with Rust news</a></li> </ol> <h2 id="you-can-t-use-anonymous-functions-for-control-flow">You can't use anonymous functions for control flow</h2> <p>In Ruby, you can <code>return</code> from almost anywhere in a function:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span style="color: #8839EF;">def</span><span style="color: #1E66F5;font-style: italic;"> find_even</span><span style="color: #E64553;font-style: italic;"> arr</span></span> <span class="giallo-l"><span> arr</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">each</span><span style="color: #7C7F93;"> {|</span><span>i</span><span style="color: #7C7F93;">|</span><span style="color: #8839EF;"> return</span><span> i </span><span style="color: #8839EF;">if</span><span> i</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">even?</span><span style="color: #7C7F93;">}</span></span> <span class="giallo-l"><span style="color: #8839EF;">end</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">#=&gt; :find_even</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>find_even </span><span style="color: #7C7F93;">[</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">2</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">3</span><span style="color: #7C7F93;">]</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">#=&gt; 2</span></span></code></pre> <p>Unfortunately, it also returns nonsense if the inputs break the invariant that an even number exists in the set:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span>return_even </span><span style="color: #7C7F93;">[</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">3</span><span style="color: #7C7F93;">]</span></span> <span class="giallo-l"><span style="color: #179299;">=&gt;</span><span style="color: #7C7F93;"> [</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;"> 3</span><span style="color: #7C7F93;">]</span></span></code></pre> <p>This is due to the expression <code>arr.each { ... }</code> returning <code>arr</code> by default.</p> <p>In Rust, the <code>return</code> keyword is much more predictable - it returns from the immediate functional scope, regardless of whether it's an anonymous function or not, meaning you can't use anonymous functions in Rust the same way you'd use block syntax in Ruby for control flow. So this code, which is roughly equivalent to the Ruby, doesn't compile:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #8839EF;">fn</span><span style="color: #1E66F5;font-style: italic;"> find_even</span><span style="color: #7C7F93;">(</span><span style="color: #E64553;">arr</span><span style="color: #179299;">: &amp;</span><span style="color: #7C7F93;">[</span><span style="color: #8839EF;">u8</span><span style="color: #7C7F93;">])</span><span style="color: #179299;"> -&gt;</span><span style="color: #8839EF;"> u8</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span> arr</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">iter</span><span style="color: #7C7F93;">()</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">for_each</span><span style="color: #7C7F93;">(</span><span style="color: #179299;">|</span><span>i</span><span style="color: #179299;">|</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #8839EF;"> if</span><span> i</span><span style="color: #179299;"> %</span><span style="color: #FE640B;"> 2</span><span style="color: #179299;"> ==</span><span style="color: #FE640B;"> 0</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #8839EF;"> return</span><span> i</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> }</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> });</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"> // Can&#39;t return null or self array, so return default 0</span></span> <span class="giallo-l"><span style="color: #FE640B;"> 0</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span></code></pre> <p>This also returns nonsense, but at least it's nonsense during compilation and not during runtime where people might actually see it:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>error[E0308]: mismatched types</span></span> <span class="giallo-l"><span> --&gt; src/main.rs:4:20</span></span> <span class="giallo-l"><span> |</span></span> <span class="giallo-l"><span>4 | return i;</span></span> <span class="giallo-l"><span> | ^ expected `()`, found `&amp;u8`</span></span> <span class="giallo-l"><span> |</span></span> <span class="giallo-l"><span>note: return type inferred to be `()` here</span></span> <span class="giallo-l"><span> --&gt; src/main.rs:4:20</span></span> <span class="giallo-l"><span> |</span></span> <span class="giallo-l"><span>4 | return i;</span></span> <span class="giallo-l"><span> | ^</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>For more information about this error, try `rustc --explain E0308`.</span></span> <span class="giallo-l"><span>error: could not compile `play` (bin &quot;play&quot;) due to previous error</span></span></code></pre> <p>This error just means that we're trying to return a <code>&amp;u8</code> from within the anonymous function consumed by <code>for_each</code>, and <code>for_each</code> doesn't expect any such returned value.</p> <p>If you find yourself in such a bind, break your higher order function out into a control flow mechanism that doesn't create a new functional scope:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #8839EF;">fn</span><span style="color: #1E66F5;font-style: italic;"> find_even</span><span style="color: #7C7F93;">(</span><span style="color: #E64553;">arr</span><span style="color: #179299;">: &amp;</span><span style="color: #7C7F93;">[</span><span style="color: #8839EF;">u8</span><span style="color: #7C7F93;">])</span><span style="color: #179299;"> -&gt;</span><span style="color: #8839EF;"> u8</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #8839EF;"> for</span><span> i</span><span style="color: #8839EF;"> in</span><span> arr</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">iter</span><span style="color: #7C7F93;">() {</span></span> <span class="giallo-l"><span style="color: #8839EF;"> if</span><span> i</span><span style="color: #179299;"> %</span><span style="color: #FE640B;"> 2</span><span style="color: #179299;"> ==</span><span style="color: #FE640B;"> 0</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #8839EF;"> return</span><span style="color: #179299;"> *</span><span>i</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> }</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> }</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #FE640B;"> 0</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span></code></pre> <p>💡 Note: llogiq mentioned that you can easily replace this with a call to <code>find()</code> to keep using functional style method chaining. Do that! This was just a contrived example.</p> <p>💡 Note #2: Oliver M. S. from LinkedIn alerted me to the <code>[core::ops::ControlFlow](https://doc.rust-lang.org/beta/core/ops/enum.ControlFlow.html)</code> trait used with <code>try_*</code> methods like <code>try_for_each</code> that generalizes this kind of thing. Holy shit, it responds to <code>?</code> like Result and Option? That's slick. 🔥</p> <h2 id="the-build-times">The build times</h2> <p>A recent <a rel="external" href="https://opensource.googleblog.com/2023/06/rust-fact-vs-fiction-5-insights-from-googles-rust-journey-2022.html">Google blog post</a> contained some great news for Rust: it's no more or no less difficult to learn than other languages. It also noted some room for improvement:</p> <blockquote> <p>"Slow build speeds were by far the #1 reported challenge that developers have when using Rust, with only a little more than 40% of respondents finding the speed acceptable."</p> </blockquote> <p>Abi Noda, a developer experience evangelist, also recently wrote <a rel="external" href="https://www.linkedin.com/pulse/build-times-developer-productivity-abi-noda">a newsletter</a> covering a study on build time effects on developer effectiveness. That study found that the relationship between build times and likelihood of a developer completing a task was linear - that is, any reduction in build times is an increase in the likelihood of completing a task.</p> <p>In my experience, the build times have been noticeable, though my overall experience is that it's less painful overall than the dice roll that is building nokogiri on a legacy Rails code base.</p> <p>Still, it's important to note that you can reduce local buildtimes by a significant proportion just by using <a rel="external" href="https://github.com/mozilla/sccache">sccache</a>. sccache is a caching mechanism by Mozilla that wraps calls to the Rust compiler and speeds up build times by caching compilation artifacts. If you're using GitHub Actions, sccache has a GitHub Actions' cache API <a rel="external" href="https://github.com/mozilla/sccache/blob/main/docs/GHA.md">backend</a>.</p> <p>Just using <code>sccache</code> in CI netted us a 18% build time improvement. Locally, I saw much better results on the order of 50% build time reductions, probably because it's not making network calls to a remote caching service.</p> <h2 id="traits-traits-traits-and-more-traits">Traits, traits, traits, and more traits</h2> <p>It's hard to overstate the importance of traits in Rust. To use Ruby as an example again, the method resolution algorithm goes roughly through these steps:</p> <ol> <li> <p>Is the method defined on the object?</p> </li> <li> <p>Is the method defined on the class of the object?</p> </li> <li> <p>Is the method defined on any parent classes?</p> </li> </ol> <p>If so, it will execute the method found on the object, class, or class hierarchy. Basically every time you ask Ruby to perform a task, it shuffles through its pant pockets, then jacket pockets, then its parents' pockets until it finds something suitable and hands it off to you.</p> <p>Rust, on the other hand, must know not just that it has the thing you're looking for, but that the thing you're interacting with conforms to the exact behavior you're looking for. Instead of asking "is the method here?" Rust asks "is this the <em>kind</em> of thing you're even looking for?" To compare:</p> <ol> <li> <p>What type of thing is this?</p> </li> <li> <p>What trait, or types of behaviors, does it implement?</p> </li> <li> <p>What trait is being requested? i.e. just because a trait defines the same methods as another one doesn't mean they are equivalent.</p> </li> </ol> <p>This is far more correct, but can be onerous to deal with. Instead of shuffling through pockets, it's more akin to registering for a passport. Your data structures are citizens of certain countries and if they apply for a passport and meet the requirements, they can travel to other countries. However, their statuses must be known before their travel privileges are. Pockets will be pre-searched by the Rustc Traits Authority.</p> <p>Traits are the passports of the Rust ecosystem. If you're using a third party library, you may need to implement a trait in order to use the library on your data structures. Alternately, a library may lean on traits already implemented in the standard library to operate. Either way, it's traits all the way down. 🐢</p> <h2 id="keeping-up-with-rust-news">Keeping up with Rust news</h2> <p>I have a short list of places I go to for Rust related news and community:</p> <ol> <li> <p><a rel="external" href="https://this-week-in-rust.org/">https://this-week-in-rust.org/</a> - this weekly newsletter is my favorite avenue for keeping up. It includes the community's blog posts, which you can submit yourself, community events (in-person and online), a quote of the week which I am especially fond of, and more. If you do nothing else, sign up for this mailing list.</p> </li> <li> <p><a rel="external" href="https://old.reddit.com/r/rust/">https://old.reddit.com/r/rust/</a> - I don't agree with where Reddit is going [<a href="https://briankung.dev/2023/07/16/rust-notes/#enshittification">0</a>] and I see a lot less activity here than I used to, but it was the most active and dynamic Rust community for a long time and a lot of good discussion has happened on this subreddit.</p> </li> <li> <p><a rel="external" href="https://users.rust-lang.org/">https://users.rust-lang.org/</a> - My new haunt for community related stuff. Go here and post! It's a lot slower, but slow can be good.</p> </li> <li> <p><a rel="external" href="https://lobste.rs/t/rust">https://lobste.rs/t/rust</a> - Lobste.rs is a technical community where a lot of good discussion happens. Also less active than Reddit, but generally the quality of discussion is better.</p> </li> <li> <p><a rel="external" href="https://news.ycombinator.com/">https://news.ycombinator.com/</a> - Of course the orange site. Rust is beyond the point where "written in Rust" means an automatic frontpage on Hacker News, and is, in terms of technology almost boring for perpetually online folk like myself, but bigger announcements will often have good technical discussions. You can count on this crowd to start fights about having a Code of Conduct, though. In terms of psychological safety, Rust project (meaning Foundation + community) moderated spaces feel much safer.</p> </li> <li> <p>Finally, just following #rustlang on Mastodon. I'd say Twitter, but I <em>*sigh*</em> can't really endorse using Twitter [<a href="https://briankung.dev/2023/07/16/rust-notes/#enshittification">0</a>].</p> </li> </ol> <p>I don't do Discord unless I have specific questions, I don't do video platforms because I don't have the time and it aggros my kids, and I also don't do podcasts because there haven't been any ongoing Rust-specific podcasts that I've liked (and I'm aware of Rustacean Station). I did enjoy <a rel="external" href="https://newrustacean.com/">New Rustacean</a>, which was about learning Rust, and <a rel="external" href="https://podcasts.apple.com/us/podcast/building-with-rust/id1553513574">Building with Rust</a>, which is a series of interviews, but they have both unfortunately come to an end, or so it seems.</p> <p>Another angle to take is not just "where," but "who." I find these people worthwhile to follow:</p> <ol> <li> <p><a rel="external" href="https://thesquareplanet.com/">Jon Gjengset</a> - A prolific YouTuber and Rust personality who is renowned for long, in-depth live coding adventures. I will admit, I have not gotten through a single live coding video. <a rel="external" href="https://www.youtube.com/playlist?list=PLqbS7AVVErFiWDOAVrPt7aYmnuuOLYvOa">Crust of Rust</a> is a great series and I <em>have</em> finished sitting through one of those, however.</p> </li> <li> <p><a rel="external" href="https://twitter.com/m_ou_se?lang=en">Mara Bos</a> - Rust Lang Library team lead and author of Rust Atomics and Locks, Mara's posts always pop up on my feeds explaining neat new features in Rust releases.</p> </li> <li> <p>Aria Beingessner / Gankra - author of <a rel="external" href="https://rust-unofficial.github.io/too-many-lists/">Learning Rust With Entirely Too Many Linked Lists</a> and author of cargo-dist and oranda at <a rel="external" href="https://axo.dev/">axo.dev</a>, Gankra has a deep understanding of Rust. Make sure to check out their blog: <a rel="external" href="https://faultlore.com/blah/">https://faultlore.com/blah/</a></p> </li> <li> <p><a rel="external" href="https://blog.burntsushi.net/about/">Andrew Gallant / burntsushi</a> - I struggled to remember burntsushi's IRL name because they're so prolific on reddit. They're the author of the regex implementation in Rust and when they comment online they're generous with benefit of the doubt and excellent at explaining themselves.</p> </li> <li> <p><a rel="external" href="https://fasterthanli.me/">Amos Wenger / fasterthanlime</a> - Amos is another incredibly prolific YouTuber and blogger. Many a time I have loaded up a new fasterthanlime article, glanced at the estimated read time, and realized I would not be finishing the article in one sitting.</p> </li> </ol> <p>There are tons of helpful people in the Rust community - this is just a sample off the top of my head.</p> <hr /> <p>[0]: <em>"Pluralistic: Tiktok's enshittification (21 Jan 2023)" - <a rel="external" href="https://pluralistic.net/2023/01/21/potemkin-ai/">https://pluralistic.net/2023/01/21/potemkin-ai/</a></em></p> 2023-05-20T17:29:09+00:00 2023-05-20T17:29:09+00:00 Unknown https://briankung.dev/2023/05/20/one-of-my-favorite-rust-features/ <p>I realized that one of my favorite Rust features doesn't have a blog post highlighting it [<a href="https://briankung.dev/2023/05/20/one-of-my-favorite-rust-features/#0">0</a>], so I thought I'd talk a bit about documentation comments and doc tests in Rust.</p> <p>I'll give you a quick example. Let's say you have some code:</p> <p>[](/assets/images/carbon1-3.png) <p>There's a <code>cargo</code> command to generate documentation from that code:</p> <p><a href="/assets/images/screenshot-2023-05-20-at-3.55.31-pm.png"><img src="/assets/images/screenshot-2023-05-20-at-3.55.31-pm.png" alt="A terminal window showing just the command `cargo doc --open`" /></a></p> <p>Which results in automatically generated documentation that looks like this:</p> <p><a href="/assets/images/screenshot-2023-05-20-at-3.57.07-pm.png"><img src="/assets/images/screenshot-2023-05-20-at-3.57.07-pm.png" alt="A screenshot of a Rust project, or crate, called &quot;play&quot; with a single struct, Puppy, with no additional description." /></a></p> <p>In any other language before learning Rust, I would have been pretty impressed at this point [<a href="https://briankung.dev/2023/05/20/one-of-my-favorite-rust-features/#1">1</a>]. The fact that documentation can be generated with a single command with the default toolchain, no added components? Pretty good already. But Rust lets you take it several steps further with documentation comments.</p> <p>You already saw the code comment in the screenshot indicating what file I was referring to. That looked like this:</p> <p><a href="/assets/images/carbon3.png"><img src="/assets/images/carbon3.png" alt="// src/lib.rs" /></a></p> <p>Rust also supports a triple slash comment like this:</p> <p>[](/assets/images/carbon2-1.png) <p>That third slash is not just decorative. Let's rerun <code>cargo doc</code> and refresh our generated documentation:</p> <p><a href="/assets/images/screenshot-2023-05-20-at-4.04.12-pm.png"><img src="/assets/images/screenshot-2023-05-20-at-4.04.12-pm.png" alt="A screenshot of a Rust project, or crate, called &quot;play&quot; with a single struct, Puppy, with the description &quot;Guaranteed to be a very cute puppy.&quot;" /></a></p> <p>Et voilà! The description has appeared in the documentation. And that's not all. You can even annotate individual fields.</p> <p><a href="/assets/images/carbon8.png"><img src="/assets/images/carbon8.png" alt="/// Guaranteed to be a very cute puppy pub struct Puppy { /// The answer to &quot;Who&#39;s a good lil puppy? 🥰&quot; pub name: String, }" /></a></p> <p>And clicking through to the detail view of the struct gives you the description as well as annotations on the fields:</p> <p><a href="/assets/images/screenshot-2023-05-20-at-4.11.24-pm.png"><img src="/assets/images/screenshot-2023-05-20-at-4.11.24-pm.png" alt="Same screenshot as earlier, but the `name: String` has a description as well: The answer to &quot;Who&#39;s a good lil puppy? 🥰&quot;" /></a></p> <p>Ah, but that's (still) not all. Rust's documentation tooling solves yet another problem. Let's say you have an method on <code>Puppy</code> like so:</p> <p><a href="/assets/images/carbon10.png"><img src="/assets/images/carbon10.png" alt="impl Puppy { pub fn whos_good(self) -&gt; String { self.name } }" /></a></p> <p>You might want to document it in a code comment like so:</p> <p><a href="/assets/images/carbon11.png"><img src="/assets/images/carbon11.png" alt="impl Puppy { // This responds with the puppy&#39;s name: // let roscoe = Puppy { name: &quot;Roscoe&quot;.to_string() }; // roscoe.whos_good() == &quot;Roscoe&quot;; // true pub fn whos_good(self) -&gt; String { self.name } } " /></a></p> <p>But let's say that <code>whos_good</code> ended up changing for some reason, for example if <code>Puppy</code> was refactored to have a <code>first_name</code> and <code>last_name</code> instead ()</p> <p>[](/assets/images/carbon15.png) <p><em>Aside: <a rel="external" href="https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/">Falsehoods Programmers Believe About Names</a> is a great read.</em></p> <p>Now our comment block is out of date! Luckily, Rust supports one more feature that helps keep code comments from going stale: <a rel="external" href="https://doc.rust-lang.org/rustdoc/write-documentation/documentation-tests.html">Documentation tests</a>. For context, doc comments are written in markdown. By default, markdown code blocks in documentation comments are <em>executed as code</em>. That's right, in Rust even your code comments are Turing-complete.</p> <p>If we rewrite our comment <em>just so</em>, we can get <code>cargo test</code> to run that assertion instead of relegating it to the dust bins of version control. Look! VS Code even uses appropriate syntax highlighting in the doc test block:</p> <p>[](/assets/images/screenshot-2023-05-20-at-4.59.32-pm.png) <p>And the test will run with the rest of the unit tests when you run <code>cargo test</code>, but for our screenshot's sake, let's just specify the documentation tests for now with <code>cargo test --doc</code> and silence the backtrace with <code>RUST_BACKTRACE=0</code>. So the full command is <code>RUST_BACKTRACE=0 cargo test --doc</code>:</p> <p><a href="/assets/images/screenshot-2023-05-20-at-5.03.14-pm.png"><img src="/assets/images/screenshot-2023-05-20-at-5.03.14-pm.png" alt="Screenshot of the output of the cargo test output, mainly to show that the doc test assertion failed with left = &quot;Roscoe McPuppyFace&quot; and right = &quot;Roscoe.&quot;" /></a></p> <p>And if we fix the documentation...</p> <p>[](/assets/images/screenshot-2023-05-20-at-5.07.49-pm.png) <p>...we fix the tests:</p> <p><a href="/assets/images/screenshot-2023-05-20-at-5.09.57-pm.png"><img src="/assets/images/screenshot-2023-05-20-at-5.09.57-pm.png" alt="cargo test --doc passing!" /></a></p> <p>And look at that slick documentation!</p> <p><a href="/assets/images/screenshot-2023-05-20-at-5.11.39-pm.png"><img src="/assets/images/screenshot-2023-05-20-at-5.11.39-pm.png" alt="Screenshot of updated Puppy struct documentation with nicely syntax highlighted code block describing the whos_good function" /></a></p> <p>I loooove Rust's documentation features. It's one of those tools that work so well that I end up <em>wanting</em> to document my code if nothing else just to admire the documentation later. I loved it so much I set up a GitHub Actions workflow at work to publish our documentation to our repo's GitHub Pages so all my coworkers could admire their handiwork as well.</p> <h2 id="bonus-github-action">Bonus: GitHub Action</h2> <p>More on that last part. I'm not going to go into too much detail here, but on your repo you can set the Page to deploy from a GitHub Actions workflow:</p> <p>[](/assets/images/screenshot-2023-05-20-at-5.20.22-pm.png) <p>Then you can use the workflow below to deploy your documentation on every merge to a branch of your choice (<code>"develop"</code> in this case):</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="yaml"><span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"># Simple workflow for deploying static content to GitHub Pages</span></span> <span class="giallo-l"><span style="color: #1E66F5;">name</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> Deploy static docs content to Pages</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #FE640B;">on</span><span style="color: #179299;">:</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"> # Runs on pushes targeting the default branch</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> push</span><span style="color: #179299;">:</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> branches</span><span style="color: #179299;">:</span><span style="color: #7C7F93;"> [</span><span style="color: #40A02B;">&quot;develop&quot;</span><span style="color: #7C7F93;">]</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"> # Allows you to run this workflow manually from the Actions tab</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> workflow_dispatch</span><span style="color: #179299;">:</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"># Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages</span></span> <span class="giallo-l"><span style="color: #1E66F5;">permissions</span><span style="color: #179299;">:</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> contents</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> read</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> pages</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> write</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> id-token</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> write</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"># Allow one concurrent deployment</span></span> <span class="giallo-l"><span style="color: #1E66F5;">concurrency</span><span style="color: #179299;">:</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> group</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> &quot;pages&quot;</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> cancel-in-progress</span><span style="color: #179299;">:</span><span style="color: #FE640B;"> true</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #1E66F5;">jobs</span><span style="color: #179299;">:</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"> # Single deploy job since we&#39;re just deploying</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> deploy</span><span style="color: #179299;">:</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> environment</span><span style="color: #179299;">:</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> name</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> github-pages</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> url</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> ${{ steps.deployment.outputs.page_url }}</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> runs-on</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> ubuntu-latest</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> steps</span><span style="color: #179299;">:</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> -</span><span style="color: #1E66F5;"> name</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> Checkout</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> uses</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> actions/checkout@v3</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> -</span><span style="color: #1E66F5;"> name</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> Setup Pages</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> uses</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> actions/configure-pages@v3</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> -</span><span style="color: #1E66F5;"> uses</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> actions-rs/cargo@v1</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> with</span><span style="color: #179299;">:</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> command</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> doc</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> -</span><span style="color: #1E66F5;"> name</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> index.html redirect</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> uses</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> &quot;DamianReeves/write-file-action@master&quot;</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> with</span><span style="color: #179299;">:</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> path</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> ./target/doc/index.html</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> write-mode</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> overwrite</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> contents</span><span style="color: #179299;">:</span><span style="color: #8839EF;"> |</span></span> <span class="giallo-l"><span style="color: #40A02B;"> &lt;meta http-equiv=&quot;Refresh&quot; content=&quot;0; url=&#39;🚧YOUR_CRATE_NAME🚧/index.html&#39;&quot; /&gt;</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> -</span><span style="color: #1E66F5;"> name</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> Upload artifact</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> uses</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> actions/upload-pages-artifact@v1</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> with</span><span style="color: #179299;">:</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"> # Upload docs repository</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> path</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> &#39;./target/doc&#39;</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> -</span><span style="color: #1E66F5;"> name</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> Deploy to GitHub Pages</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> id</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> deployment</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> uses</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> actions/deploy-pages@v1</span></span></code></pre> <p>Because the documentation is stored in <code>YOUR_CRATE/target/doc</code> and there's no top-level <code>index.html</code>, we need to create one that simply redirects to your crate's documentation. That's what the <code>DamianReeves/write-file-action@master</code> action is for.</p> <p>GitHub should give you a short url at <code>https://YOUR_ORG_NAME.github.io/YOUR_REPO_NAME</code> which will redirect you to some Heroku-style subdomain. You can even limit the visibility to contributors-only.</p> <p>That's it! Hope you enjoy Rust's documentation features as much as I do.</p> <hr /> <p>[<a href="https://briankung.dev/2023/05/20/one-of-my-favorite-rust-features/#-0">0</a>]: Apparently this is all covered in the guide to "<a rel="external" href="https://doc.rust-lang.org/cargo/reference/publishing.html">Publishing on crates.io</a>"</p> <p>[<a href="https://briankung.dev/2023/05/20/one-of-my-favorite-rust-features/#-1">1</a>]: There is also Javadoc, Pydoc, and YARD (Ruby) docs for generating documentation.</p> 2023-03-04T22:25:31+00:00 2023-03-04T22:25:31+00:00 Unknown https://briankung.dev/2023/03/04/public-school-is-a-bad-deal-for-us/ <p><em>Actually, that really understates the problem - we feel that living in the USA is a bad deal for families in general, but for now I'll focus specifically on public school and daycare.</em></p> <p>My wife and I were taking a stroll around the neighborhood with our three kids during the week when I realized - there's no one else out here. As in, there are no other families enjoying the (at the time) fall weather. Occasionally you would see a caretaker pushing a stroller, or a pair of adults power walking, but we were basically the only nuclear family out on the sidewalks. And we were almost always the only nuclear family we saw on weekdays.</p> <p>My mind took me to the fact that the fault lines of our society are all determined by fealty to corporate interests. After all, the biggest reason for putting kids in daycare, especially at such young ages, is to be able to work.</p> <p>Anyway, that triggered this diatribe on the downsides of sending our kids out to daycare or public school. The caveat is that both my wife and I have the privilege to be able to work remotely, support ourselves, and also raise our kids - even if it's somewhat hectic at times.</p> <p>⚠️ To reiterate: this essay is <em>especially</em> true for us because we can work remote with reasonable work-life balances - to the point where we feel like we would prefer to keep the kids home for now. This is not everyone's situation, but the downsides I list below for families living in America are, in my opinion, downsides for all.</p> <p>So let's go one by one through reasons why people send their kids to school - "school" meaning daycare on up, starting with the hygiene hypothesis, or how people have interpreted the hygiene hypothesis, anyway. Basically, the idea that sending your kids into the line of fire builds up their immunity against pathogens.</p> <h2 id="immunity">Immunity</h2> <p>When the pandemic hit, we withdrew our eldest from the school system and began what eventually turned into an ad hoc system for homeschooling.</p> <p>Now that President Biden has <a rel="external" href="https://www.npr.org/2022/09/19/1123767437/joe-biden-covid-19-pandemic-over">declared</a> covid "over," but more to the point that all of our kids are vaccinated against it and we're busy with work, we have been considering when and how to get the kids back into the public sphere. That means daycare and school.</p> <p>Immediately the next thing that happened was a wave of RSV, flu, <em>and</em> covid. If you're not a parent or you're not familiar with RSV, it's called <a rel="external" href="https://www.cdc.gov/rsv/index.html">Respiratory Synctial Virus</a>, and in my circles it's put more kids into the ER than covid has. Here's an apocalyptic article on the trio of diseases: <a rel="external" href="https://www.nytimes.com/2022/11/01/science/rsv-children-hospitals.html">US Children’s Hospitals Are Overwhelmed by RSV - The New York Times</a> (note: I would treat "immune debt" as a more questionable idea than the article presents it, which I cover <a href="https://briankung.dev/2023/03/04/public-school-is-a-bad-deal-for-us/#immune-debt">below</a>).</p> <p>Like clockwork, parents in our circles had to call off work to take care of their kids. It's a miserable experience - oftentimes you have to take care of your children while you yourself require care-taking due to being sick with the same thing. And it's almost inevitable if you send young kids to school. Even older kids will share food with each other and swap masks, if they even bother to wear masks.</p> <p>So how do you protect your kids and yourself if you send your kids into public school? You don't. You just accept that you'll get sick. That's how it was for us before the pandemic. We would be miserable every other week. Isolating from society during the pandemic was hard, but not being sick twice a month was a revelation - one we don't want to give up.</p> <p>What about "building up the kids' immunity" you ask? Well, we know that herd immunity <a rel="external" href="https://www.statnews.com/2022/03/25/how-we-got-herd-immunity-wrong/">doesn't really work for covid</a>, reinfection after vaccination or illness is <a rel="external" href="https://www.news-medical.net/news/20220809/Icelandic-study-suggests-SARS-CoV-2-Omicron-reinfection-more-common-than-previously-thought.aspx">fairly common</a>, and reinfections <a rel="external" href="https://www.forbes.com/sites/roberthart/2022/11/10/covid-reinfections-may-boost-chances-of-death-and-organ-failure-study-finds---and-the-risk-increases-each-time-you-catch-it/?sh=57b9102b2b2d">boost chances of death and organ failure</a>. And while these articles are all about covid, it's not too far a leap to guess that other viral diseases might have similar profiles. For instance, RSV immunity <a rel="external" href="https://pubmed.ncbi.nlm.nih.gov/2010624/">lasts about 2 months</a>. Measles actually robs the people it infects of immunity <em>from other viruses</em>.</p> <p>So - they hygiene hypothesis is one thing. Dirt, bacteria, wind, rain, and sunshine are all fair game for priming a young immune system. But viruses can do some really deep damage - like <a rel="external" href="https://www.mdanderson.org/publications/focused-on-health/7-viruses-that-cause-cancer.h17-1592202.html">cause cancer</a>. If you could really exercise your immune system like you could a muscle everyone would start out licking public school drinking fountains and some would eventually graduate to licking <a rel="external" href="https://en.wikipedia.org/wiki/Biosafety_level">BSL</a> 4 door knobs. My point being that there is a limit to the progressive overload that the immune system can take.</p> <p>Viruses are also often attenuated, or made weaker, by overly <a rel="external" href="https://en.wikipedia.org/wiki/Virulence">virulent</a> strains of virus killing their hosts and therefore becoming evolutionary dead ends, whereas weaker strains continue to let their hosts spread the infection. We don't have that natural attenuation in the United States because our health care system is really good at two things: keeping you in debt, and keeping you alive. So there is a really high theoretical threshold to a virus keeping itself in check via virulence because the ER will pump you full of steroids, prescribe a raft of antibiotics (just in case), and then mail you exorbitant medical bills for the privilege over the course of the next few years.</p> <p>And we are breeding these particularly potent viruses in daycares and public schools.</p> <h2 id="socialization">Socialization</h2> <p>Another factor that people, and even doctors, bring up is that of socialization. As in, if you don't send your kids to school, they will grow up to be some sort of Neanderthalic reject, unable to cope with the modernity of the world.</p> <p>Our kids are mostly too young to be able to tell, but I have my doubts. When we sent our eldest into daycare, she socialized, but it was, ah, how do you say this sensitively...it's downright abusive.</p> <p>Our daughter, Avery, is an endlessly cheery girl, to the point where it's detrimental to her own health. Her "best friend" at the time, of which she had many, was a girl named Rylee (sp?) who would show her affection by boxing Avery's ears or pinching her upper lip so hard it would draw blood. And that's just what we heard about. When we asked the daycare staff about it, they were very apologetic and issued an incident report and did essentially nothing to prevent it from happening again, which it did.</p> <p>There are never enough teachers, because again the point of a daycare is to make a profit, not to optimize your kids' well-being, and when they are there, they are powerless to prevent anything. I wouldn't be surprised if the parents of a bully are more likely to be bullies themselves, so an educator puts their careers on the line by disciplining bullies, unless the entire administration is willing to go to bat for the teacher. Much cheaper to just find an excuse for the bully's behavior.</p> <p>And if your kid isn't one of the bullies to begin with, they might learn to be one. Avery came home from daycare at some point and started kicking her grandfather because "the boys did it."</p> <p>Our daughter isn't our only point of reference, either. Cici and I both grew up in the US school system. We are familiar with the abuses one can receive from classmates and teachers. And many of our friends' children have had run-ins with bullies, racists, and racist bullies. Maybe if you're white with Aryan features, public school is a breeze for you. But for many people, and especially people of color, "socialization" means getting used to being abused.</p> <p>Our school system leads to an arms race in bullying, a Lord of the Flies environment where the bullies rule, the bullied grow resentful until they finally strike back. With the easy availability of weapons, it's no wonder that the United States is ground zero for school shootings, worldwide.</p> <h2 id="education">Education</h2> <p>I've treaded and retreaded this so many times that it's a given to me, but I do not believe that children learn better in school, at least early on. Both in personal experience with our eldest and the fact that schools <a rel="external" href="https://www.cdc.gov/sleep/features/schools-start-too-early.html">start too early</a>, disrupting sleeping schedules, or that we wouldn't be able to <a rel="external" href="https://briankung.dev/2022/04/13/how-and-why-our-kids-are-learning-chinese/">teach our children Chinese</a>, or teach them about the racism that we've faced in various institutions - school included.</p> <p>📚 Our personal experience with having our eldest study math or Chinese is that, given the right incentives, she can focus for longer periods of time on difficult subjects than she likely would be asked to in school. While we started her out with fun apps for learning math (like <a rel="external" href="https://dragonbox.com/products">DragonBox</a> and later <a rel="external" href="https://www.prodigygame.com/main-en/">Prodigy</a> games), we've since moved on to more focused study using <a rel="external" href="https://www.ixl.com/">IXL</a>, and she's done really well. Similarly for Chinese, we started her out on the <a rel="external" href="https://www.ihuman.com/website/pc/en/shizi/">iHuman</a> 洪恩识字 app and then graduated her to reading short fiction and writing messages to her grandparents.</p> <p>Let's also consider the effect that the school system has on kids' outlooks on learning. I think the biggest problem with keeping the kids busy in school is the busy-ness. They have a lot of <em>things</em> to do and not enough time to let their minds wander or get bored and do things just because. It's modeled after work.</p> <p>We consider boredom to be an essential part of learning. This ties into our policy on technology, as well, but essentially outside of a few focused hours of work time, Avery is free to do whatever she wants - except screen time. As a result, she is often bored, and as a result of <em>that</em>, she:</p> <ul> <li> <p>Plays with her siblings</p> </li> <li> <p>Crafts using cardboard and knick-knacks around the house</p> </li> <li> <p>Does origami</p> </li> <li> <p>Plays the piano</p> </li> <li> <p>Reads every new fiction book she can get to</p> </li> <li> <p>Reads extensively about flowers and succulents</p> </li> </ul> <p>We don't have to push her to do anything artistic or literary. She just wants to. And, importantly, she has the time to do it.</p> <p>School busy work is often tedious, uninspiring, and time consuming. Boredom is merely tedious until you find the next thing to get in trouble with.</p> <p>To be fair, a lot of the downsides are mediated by the kids' relationships with their parents. School doesn't have to be a tedious, boring slog, but it does end up taking freedom away from the kids, by design (otherwise they'd be interrupting your standups and 1:1's).</p> <h2 id="just-plain-bad">Just plain bad</h2> <p>Forgot to mention a few key elements that play into our decision to keep the kids at home that are just plain bad instead of replies to common reasons to send the kids to school:</p> <h3 id="school-shootings-and-active-shooter-drills">School shootings and active shooter drills</h3> <p>Despite the fact that it is statistically unlikely for your child to be actually shot in a school shooting, the fact that it's likely <em>enough</em> for active shooter drills to be taught in school is terrifying. The drills themselves should be considered traumatizing and the fact that we allow this to happen at all is a damning condemnation of where we are at as a country. And if an active shooter situation ever does happen, it's not just the shooting victims who are traumatized, it's the entire school. So we view school shootings as likely to affect us and growing more likely with every year, compared to when my wife and I were in school.</p> <p>Even if the drills themselves <em>weren't</em> traumatizing, the things we've heard of the active shooter drills themselves would be hilarious if it weren't so serious. For example, our friend's kid was given the advice to "zig zag to avoid fire" as if it was a big game of Counter-Strike. That is a great way to incite uncontrollable chaos in a class of kindergartners. "Hide behind your classmates' bodies and play dead" can be practical, yet horrifying.</p> <p>These things shouldn't happen. Ever. Having to think about school shootings alone makes the US a horrible place to raise a family.</p> <h3 id="meal-habits">Meal habits</h3> <p>On a less acutely deadly note, children simply don't eat well at school, and it is unacceptable to us. My nephew, who is in high school, is given 20 minutes to eat lunch and barely eats more an apple. Younger children don't eat, and when they do they do their damnedest to infect each other with the latest and greatest viruses.</p> <p>At home, we make sure the kids are well fed, even if it takes cajoling, bribing, and presenting different options at the right times. If our eldest is any indicator we will eventually have a bunch of great eaters, but right now it is a game of escalating bribery. It would pain us to no end to send the kids to school with a full lunch box and then have to throw most of it out when they come back. It's bad for their health and it's food waste.</p> <hr /> <p>In the end, school is a really tough deal for us right now. We're still struggling with our eventual reentry into the world and what's best for our kids, because while this environment is ideal for them in some ways, it's not a typical one, and we don't want that to end up stunting their ability to adapt to the world we live in.</p> 2023-02-17T17:25:20+00:00 2023-02-17T17:25:20+00:00 Unknown https://briankung.dev/2023/02/17/professional-rustacean-3-months-in/ <p>Hi! 👋 I did end up landing that Rust job I was <a rel="external" href="https://briankung.dev/2022/08/18/contributing-to-the-vector-remap-language/">looking for</a>, and I've been wrestling with the Rust programming language now for a good couple of months - not quite 3, if I'm to be honest, but who's counting.</p> <p>To be honest, I'm a little bit ambivalent. I think it's a great language for a few reasons, but I will be honest in that I am not nearly as productive as I would be in other languages. For context, I'm using it mostly in web development, where my language and framework of choice is usually Ruby and Rails.</p> <p>Yes, I am aware of the opinion <a rel="external" href="https://news.ycombinator.com/item?id=34835631">on Hacker News</a> that it's a good language for low level use cases, but arguably not the right choice for web development. Then you have comments like <a rel="external" href="https://news.ycombinator.com/item?id=34836459">this one</a> that run counter to that narrative. I have to wonder why Rust seems to work at all levels for some, but not as much for me, but then I only have 2-3 months of experience with the language.</p> <p>Anyway, these aren't meant to be pros or cons, just notes.</p> <h2 id="it-s-programmer-catnip">It's programmer catnip</h2> <p>I would guess that <a rel="external" href="https://www.scientificamerican.com/article/experts-how-does-catnip-work-on-cats/">about 70-80% of programmers</a> love rolling around in it, stuffing our faces in it, and playing with the language features. The reason is pretty simple - we're generally technical people, and Rust has the wonderful characteristic of exposing us to all the technicalities underlying the systems we're building, but with warning signs.</p> <p>When I use Rust, I feel like I'm learning about the underlying systems and models. To take a cheap example, consider Rust's <a rel="external" href="https://runrust.miraheze.org/wiki/String_Types">string types</a>. The fact that they are modeled in such a complex way <em>reflects</em> the complexity of string handling in computers. Learning how to use strings in Rust teaches you the difference between strings that are valid UTF-8, operating system strings, C-style strings, and the difference between borrowed and owned strings.</p> <p>Does it suck? Yes. Is it educational? Also yes. Is it gratifying to eventually understand? Yes.</p> <p>Learn to use Rust and you'll learn about computing. And that's a big draw to a lot of programmers, because there's a big overlap between programmers and people who, well, like computers.</p> <h2 id="type-system-gymnastics">Type system gymnastics</h2> <p>In Ruby, where everything is an object, you often perform message passing gymnastics to glue everything together. If it quacks like a duck but you need it to walk like a human, well, you better get the duck quacking to the dog that barks at the human. You can inject anything anywhere, as long as it passes the right messages. It's like infinitely interchangeable gears.</p> <p>This doesn't fly in Rust. Rust is not object oriented. Rust is more type oriented than object oriented. Just because two objects have functions with the same name doesn't mean that they're interchangeable. Things hinge more on whether they are the same type, or can be coaxed into that type, or implement the same trait. So instead of mashing objects together willy nilly, you need to figure out how to get from one type or implementation to another.</p> <p>In Ruby, you can traverse the world via methods. In Rust, you must traverse the world via the type system. It's a different trapeze, but you can still swing it.</p> <h2 id="productivity-tradeoff">Productivity tradeoff</h2> <p>I'm way less productive in Rust than I am in Ruby, at least for web development purposes. I am a lot more confident in the quality of the code, though. Oftentimes when I find myself attempting to write unit tests, I write one as a higher level integration test to check that the method I'm adding exists and behaves, but then any additional tests that I would have written in Ruby to handle edge cases are instead handled by the type system and simply refuse to compile.</p> <p>That said, I am 2.5 months into this Rust journey and the type system gymnastics can definitely take a toll. But it's such potent catnip that I find myself energized to come back to it again and again.</p> <p>I think that confidence is invisible to business stakeholders, but I suspect that, despite the lower <a rel="external" href="https://en.wikipedia.org/wiki/Velocity_(software_development)">initial velocity</a>, there's much higher guarantee of quality and maintainability.</p> <p>First, the Rust compiler probably won't stop compiling your application code, at least for any code written in recent times, due to <a rel="external" href="https://blog.rust-lang.org/2014/10/30/Stability.html">Rust's stability guarantee</a>. <a rel="external" href="https://www.reddit.com/r/rust/comments/11348jp/examples_of_old_ca_100_rust_code_that_still/">Probably</a>. That's just not true in Ruby. Breaking changes with no <a rel="external" href="https://doc.rust-lang.org/edition-guide/editions/index.html">edition system</a> mean that the chronological window for valid code shifts. Ancient code in other languages will bitrot faster than in Rust.</p> <p>For instance, one of our Ruby code bases dates back to Christmas, Dec 2018 at the earliest, and no longer builds. Meanwhile, from the linked Reddit post above:</p> <blockquote> <p><a rel="external" href="https://crates.io/crates/suffix/0.2.10"><code>suffix 0.2.10</code> was published on April 15, 2015</a>, one month <em>before</em> Rust 1.0 came out [and still compiles]</p> <p>- <a rel="external" href="https://www.reddit.com/r/rust/comments/11348jp/examples_of_old_ca_100_rust_code_that_still/">Examples of old (ca. 1.0.0+) Rust code that still compiles? : rust</a></p> </blockquote> <p>That said, I'm still slower to write Rust than I am to write Ruby. Perhaps that's due to a lack of familiarity with the language in practice - we're measuring a decade or more of experience with Ruby with maybe a few months of professional experience with Rust.</p> 2022-12-07T21:19:42+00:00 2022-12-07T21:19:42+00:00 Unknown https://briankung.dev/2022/12/07/ai-morality/ <p><em>I cobbled this together from a few comments I'd made on Facebook - please forgive the lack of coherency.</em></p> <hr /> <p>Is it moral to ask AI art program to create something "in the style of" a living artist?</p> <p>Is it moral to ask an AI art program to create something in between styles of two artists?</p> <p>What about three artists? Four?</p> <p>Stable Diffusion and other AI art bots take this to the nth degree. Just because the number, n, of artists is very high does not mean that it is not immoral to profit off of the work of artists. And remember - there are people who are using these artists' work to get rich.</p> <p>Also note that just because this usage of public domain hasn't been challenged in the court of law doesn't mean it never will be.</p> <p>I think this is a brand new day for IP law.</p> <p>And for those who might argue that "impressionism or cubism isn't protected by law," it's not so much that the <em>style</em> of art is protected, but that the works made in those styles have been directly and algorithmically derived from artists without their consent. Today it's "make a comic in the style of Shen Comix" and tomorrow it'll be "make me a video of Jacob saying he hates tacos" - or worse. But whether it's people's public images, the works they put online, or data that's scraped, I think the law will need to catch up in protecting the data that we produce and the impact of its derivations on us.</p> <p>Another way to think of this problem is that intellectual property is supposed to protect the rights - and properties - of the inventors. In the case of AI produced properties, who is the inventor of the work? The person who wrote the prompt? The AI that responded? The owners of the data that it was trained on? Or the author of the algorithm that produced the AI model?</p> <p>Either way, I think it's a valid argument to make that the owners of the training data should have a say in this.</p> <p>I want to acknowledge that AI is really exciting and out of everyone in my circles, I've probably used it the most. I've used Stable Diffusion on my Mac to produce images and I've also used ChatGPT to produce code, or as a pair programming partner. The technology is really exciting and magical.</p> <p>But I think we all underestimate what we're owed here. Researchers have trained these bots on our data - likely our posts on Facebook, my public blog and open source code have all been used to train various AI. Media artists may have the most skin in the game here, with the amount of labor and the sheer amount of data they produce.</p> <p>This is a data issue, but ultimately it is a labor issue. All the data we produce - and we produce it constantly simply through the labor of existing - should be our intellectual property and should be protected under the law. These kinds of AI wouldn't be possible in jurisdictions with strong data privacy laws like the EU and California.</p> <p>And it shouldn't be, because the ill gotten gains of those data violations end up, ultimately, in the hands of, like, three people who have been chosen mostly by privilege, racism, and heritage to amass wealth.</p> <p>Now if there was an AI model that was trained using *only* fully consenting users' data† and that model was free for everyone to use everywhere, I'd have a lot less to complain about. However, Midjourney, Lensa, ChatGPT, et al. are not that. Everyone should read Karla Ortiz's post about the moral issues surrounding the image databases that were used to train Stability AI, Midjourney, and even Google: <a rel="external" href="https://threadreaderapp.com/thread/1593723972314882048.html">https://threadreaderapp.com/thread/1593723972314882048.html</a></p> 2022-12-03T17:42:28+00:00 2022-12-03T17:42:28+00:00 Unknown https://briankung.dev/2022/12/03/fun-with-gleam/ <p>This'll be a short blog post, but Gleam (v0.25.0) is an Erlang OTP language, akin to Elixir (in fact, recently added full compatibility with Elixir packages). It's been <a rel="external" href="https://gist.github.com/briankung/ad1405bc7b49daff73e8f40f84445ba2">fun so far</a>.</p> <p>I think the error messages are great and the functional style language is a lot of fun to wrap my brain around. I have had some clerical issues trying to find resources for things as simple as opening files and referencing type variants, but the discord channel has been really helpful.</p> <p>For reference, opening files is apparently done through an Erlang bridge. First add <code>gleam_erlang</code> to your project:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="shellscript"><span class="giallo-l"><span style="color: #1E66F5;font-style: italic;">$</span><span style="color: #40A02B;"> gleam add gleam_erlang</span></span></code></pre> <p>Then use it like so:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="gleam"><span class="giallo-l"><span style="color: #8839EF;">import</span><span> gleam</span><span style="color: #179299;">/</span><span>erlang</span><span style="color: #179299;">/</span><span>file</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #8839EF;">pub fn</span><span style="color: #1E66F5;font-style: italic;"> main</span><span>() {</span></span> <span class="giallo-l"><span> file.</span><span style="color: #1E66F5;font-style: italic;">read</span><span>(</span><span style="color: #40A02B;">&quot;./input/two.txt&quot;</span><span>)</span></span> <span class="giallo-l"><span>}</span></span></code></pre> <p>Thanks to <a rel="external" href="https://markholmes.org/">Mark Holmes</a> for assisting me on the <a rel="external" href="https://discord.gg/Fm8Pwmy">Gleam Discord</a> with that tip!</p> <p>And referencing type constructors, or what look suspiciously like Rust's Enum variants, after import can be confusing.</p> <p>If you have a custom type like so:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="gleam"><span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">// src/animal.gleam</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">// 👇 this is a &quot;type&quot;</span></span> <span class="giallo-l"><span style="color: #8839EF;">pub type</span><span style="color: #DF8E1D;font-style: italic;"> Move</span><span> {</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"> // 👇 these are &quot;type constructors&quot;</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Rock</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Paper</span><span> </span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Scissors</span></span> <span class="giallo-l"><span>}</span></span></code></pre> <p>And I was trying to reference it elsewhere like this:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="gleam"><span class="giallo-l"><span style="color: #8839EF;">import</span><span> move.{</span><span style="color: #DF8E1D;font-style: italic;">Move</span><span>}</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #8839EF;">pub fn</span><span style="color: #1E66F5;font-style: italic;"> main</span><span>() {</span></span> <span class="giallo-l"><span> move.</span><span style="color: #1E66F5;font-style: italic;">compare</span><span>(</span><span style="color: #DF8E1D;font-style: italic;">Move</span><span>.</span><span style="color: #DF8E1D;font-style: italic;">Rock</span><span>, </span><span style="color: #DF8E1D;font-style: italic;">Move</span><span>.</span><span style="color: #DF8E1D;font-style: italic;">Paper</span><span>)</span></span> <span class="giallo-l"><span>}</span></span></code></pre> <p>You will get an error. That is because types and type constructors both importable at the same level. So you *can* import <code>Move</code> as the umbrella type for <code>Rock</code>, <code>Paper</code>, and <code>Scissors</code>, but if you're just going to use <code>Rock</code> and <code>Paper</code>, then you can do so like this:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="gleam"><span class="giallo-l"><span style="color: #8839EF;">import</span><span> move.{</span><span style="color: #DF8E1D;font-style: italic;">Rock</span><span>, </span><span style="color: #DF8E1D;font-style: italic;">Paper</span><span>}</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #8839EF;">pub fn</span><span style="color: #1E66F5;font-style: italic;"> main</span><span>() {</span></span> <span class="giallo-l"><span> move.</span><span style="color: #1E66F5;font-style: italic;">compare</span><span>(</span><span style="color: #DF8E1D;font-style: italic;">Rock</span><span>, </span><span style="color: #DF8E1D;font-style: italic;">Paper</span><span>)</span></span> <span class="giallo-l"><span>}</span></span></code></pre> <p>The other thing I was hoping for was to be able to destructure function arguments, but alas twas not to be. I had:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="gleam"><span class="giallo-l"><span>list.</span><span style="color: #1E66F5;font-style: italic;">map</span><span>(</span><span style="color: #8839EF;">fn</span><span>(</span><span style="color: #E64553;font-style: italic;">game: </span><span style="color: #DF8E1D;font-style: italic;">List</span><span>(</span><span style="color: #DF8E1D;font-style: italic;">Move</span><span>)) {</span></span> <span class="giallo-l"><span style="color: #8839EF;"> let</span><span> [a, b] </span><span style="color: #179299;">=</span><span> game</span></span> <span class="giallo-l"><span> move.</span><span style="color: #1E66F5;font-style: italic;">compare</span><span>(a, b)</span></span> <span class="giallo-l"><span>})</span></span></code></pre> <p>...and I really wanted to be able to get rid of the <code>let</code> statement like so:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="gleam"><span class="giallo-l"><span>list.</span><span style="color: #1E66F5;font-style: italic;">map</span><span>(</span><span style="color: #8839EF;">fn</span><span>([a, b]) {</span></span> <span class="giallo-l"><span> move.</span><span style="color: #1E66F5;font-style: italic;">compare</span><span>(a, b)</span></span> <span class="giallo-l"><span>})</span></span></code></pre> <p>Which is unfortunately not currently possible.</p> <p>I would say these are really minor, though. The benefit of using Gleam is that you can depend on types when you want with an Elixir-like paradigm. The error messages are great, at one point reminding me when I was trying to use <code>result.is_ok()</code> that Gleam is not an object oriented language, and that I would probably be better off using <code>Result.is_ok(result)</code>.</p> <p>Pipes are also great. I feel like I'm making Super Mario Maker with all the pipes I'm putting in. For instance, you could write the above <code>Result</code> code like so:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="gleam"><span class="giallo-l"><span>result</span></span> <span class="giallo-l"><span style="color: #179299;">|&gt;</span><span style="color: #DF8E1D;font-style: italic;"> Result</span><span>.</span><span style="color: #1E66F5;font-style: italic;">is_ok</span><span>()</span></span></code></pre> <p>Gleam is smart enough to use the result of the last computation as the first argument in the next one. But if you want, you can also reference the last result as an underscore in case you need to pass it in to a different positional argument:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="gleam"><span class="giallo-l"><span style="color: #FE640B;">100</span></span> <span class="giallo-l"><span style="color: #179299;">|&gt;</span><span> int.</span><span style="color: #1E66F5;font-style: italic;">random</span><span>(</span><span style="color: #FE640B;">0</span><span>, </span><span style="color: #7C7F93;font-style: italic;">_</span><span>)</span></span></code></pre> <p>There are a bunch of other cool things about the language but I have to go because my children are hangrily destroying things. Oh! The LSP server and automatic todo annotations are awesome. Okay, gotta go.</p> <p>Have a terrifying AI generated image:</p> <figure> <p><a href="/assets/images/000000_866775386_kdpmpp2m15_ps7.5_an_8-bit_pixel_of_mario_jumping_through_silicon_motherboard_pipes_for_the_hero_image_of_a_blog_post_generated.jpg"><img src="/assets/images/000000_866775386_kdpmpp2m15_ps7.5_an_8-bit_pixel_of_mario_jumping_through_silicon_motherboard_pipes_for_the_hero_image_of_a_blog_post_generated.jpg" alt="" /></a></p> <figcaption> <p>Stable diffusion prompt: an 8-bit pixel of Mario jumping through silicon motherboard pipes for the hero image of a blog post</p> </figcaption> </figure> <p>PIPES</p> 2022-11-14T22:14:40+00:00 2022-11-14T22:14:40+00:00 Unknown https://briankung.dev/2022/11/14/on-interviewing/ <p>Many people (millennials, at least) have been trained to kiss ass when we're interviewing, to basically "fake it til you make it." This can take several forms:</p> <ul> <li> <p>People pleasing</p> </li> <li> <p>Professional ambition</p> </li> <li> <p>Sacrificing personal boundaries</p> </li> </ul> <p>Many of these behaviors actually hurt candidates during interviews, especially marginalized candidates and young people early in their careers.</p> <p>These are behaviors that reduce the signal around whether the candidate will fit with the team. The interviewer's confidence in your person is one of the most important qualities in an interview, arguably beyond your qualifications for the role. If your resume is a dead ringer for the role but the interviewer can't be sure whether you’re actually who you say you are, what your motivations are, or what your personality is like, it's going to be a hard sell.</p> <p>And for the most part, hiring managers can figure out when someone is being authentic, has a genuine interest in the position, and is trying to figure out if the position matches their needs. It's easier to ascertain whether there's real alignment when candidates are open about their own requirements.</p> <p>On the other hand, a candidate who is trying to say the right things in order to get the job is harder to get quality information about. They might actually be qualified, but it's harder to tell because the candidate is trying to guess what the interviewer wants. When this is obvious it's damaging, and for many early career applicants it's fairly obvious. Behaviors like showering the company with faint praise, dissembling, and never disagreeing, pushing back, or showing vulnerability.</p> <p>Interviewers must also make judgments beyond the candidate's basic competency. Will they stick around? Suddenly drop out of contact? What's their work style? It can be difficult to read between the lines if they're putting up a facade. This once again lowers the confidence the interviewer can have in their decision making process.</p> <p>For their part, candidates should also ask questions about their own requirements, but generally younger people won't because they haven't established work-life boundaries. So they people please instead, or talk about how they are ambitious and eager to learn and grow, work overtime or without pay. Let's talk about each of those in turn:</p> <ul> <li> <p>People pleasing - lowers the confidence in the interviewee being entirely truthful</p> </li> <li> <p>Professional ambition - the candidate might outgrow the position which might not be a benefit to the company</p> </li> <li> <p>Sacrificing personal boundaries - could telegraph unprofessionalism, disorganization, or a bad culture fit. Abusive employers might take this as a good sign, though.</p> </li> </ul> <p>Any of these might disqualify the candidate from consideration which would exacerbate their lack of professional experience.</p> <p>And we can't forget that the power dynamic will almost always be tilted in favor of the employer. So I don't blame candidates for choosing not to be vulnerable, especially if they're early in their careers or from a marginalized background. And here's really where the crux of the matter lies - most of the power in the USA is concentrated in capital, in the business's hands. As much as everyone deserves to have a living wage (and <a rel="external" href="https://en.wikipedia.org/wiki/Universal_basic_income">universal basic income</a>!), ultimately everyone is forced to produce under capitalism.</p> <p>What does that mean for interviewees? Well, it means that people early in their careers, people from marginalized backgrounds, people who are economically disadvantaged, and many others will appear less palatable than those who were already born into privilege. They do not fit in, or they can't perform authenticity, or don't seem "professional." This extends to the first impression - the resume. While a resume may seem like a culmination of one's own efforts, it can also be seen as a list of privileges that have been afforded to you, a snowball of privilege that helps roll up the next opportunity.</p> <p>And then the problem compounds because they will have less life and professional experiences to draw from for the next interview, or they will have experiences that others deem unsuitable for the same reasons. Think of someone suing an employer for being racist and then being seen by future employers as the problem instead of their previous employer.</p> <p>Anyway, I'm a bit far afield, but this all came from several conversations with my wife, who has been hiring for her team and finding it really hard to find people who engage in an honest conversation, especially young candidates. We believe that every candidate - every person - deserves a living wage, but she can't hire them all. And we see a lot of young people making the same mistakes that we did - people pleasing and playing up strengths that are irrelevant to the job description.</p> <p>For instance, sometimes ambition is valued. In my wife's case, and we suspect in many other cases, it is not. Hiring managers don't want a candidate to be everything their company could ever want now or in the future - they want someone to fill a present need, and oftentimes that need will not go away, so if a candidate telegraphs that they will outgrow the position it's potentially disqualifying.</p> <p>(Side note, as RPG players, my wife and I shared this misconception about employment early in our careers, too - that you can never outgrow a position. If you're level 100 of course you can do level 1 work! But employers disagree; you can definitely be overqualified because you're a flight risk or odd goods.)</p> <p>So something that my wife and I both landed on in our careers is that it's easier to be authentic about our own needs and wants and push back on what we perceive as the business's requirements. If there's a lack of mutual respect or alignment, then it's a quick way to find out that it's not a good fit. However, we have the advantages of being in the middle of our careers in highly demanded fields.</p> <p>I suspect that, for many, it starts early. Children's team sports, for example. I think social knowledge and the urge to contribute on an organizational level can get a good start here. Another one is just exposure to professional settings. The push and pull of an interview, not just as a rubric for the employer, but also for the employee, requires a level of comfort that doesn't come to many people naturally without practice.</p> <p>(Incidentally, these types of things are coded as very white to me. The most white-adjacent people of color I know are all very into sports, and it's probably an advantage in your career because you learn how to socialize and organize for a group of people. Just another quiet advantage that white people take for granted, like 529s and legacy admission.)</p> <p>But that said, is it possible to teach confidence and authenticity to other job seekers, especially younger folks?</p> 2022-10-25T15:36:43+00:00 2022-10-25T15:36:43+00:00 Unknown https://briankung.dev/2022/10/25/getting-into-tech/ <p>When I graduated from college with a degree in East Asian Languages and Culture from the University of Illinois at Urbana-Champaign in 2011 (one year late), I had no idea what I was going to do. To compare with my fellow EALC majors, one had started a restaurant, one got into horseback riding, and another was a brand ambassador. So basically, I was at square one.</p> <p>A few things from my background contributed to me landing in software, however:</p> <ul> <li> <p>🕹 I was a computer enthusiast from a young age, whether that was begging my uncle to start up Commander Keen on his computer, being the first kid on the block with cable internet, or the first with a high end graphics card (all hail the Radeon 9700 Pro). I read MaximumPC to drool over the latest game graphics and that introduced me to Slashdot, where I learned about up and coming startups and also comment karma systems.</p> </li> <li> <p>💻 I picked up Learn HTML in 24 Hours at some point and wrote some basic websites - mostly unordered lists. Then I grew bored. As a result, my websites to this day are still mostly unordered lists and that's why <em>this</em> website runs on Wordpress.</p> </li> <li> <p>🐍 I also picked up a book on learning Python in high school, probably mostly because snakes are <em>cool</em> and installing software had this mythological status. Installing software was fun for the sake of enabling new things. That includes updates to software. I no longer have this fascination, but it's a nostalgic feeling. Oh, and Python? I did a few simple math problems in the REPL, like <code>1 + 1</code>, but lost interest. This was the same as my TI-83, but without the graphs! I was almost disgusted at how useless it was. Little did I know. My impression of Python remained, however. Sorry Pythonistas.</p> </li> <li> <p>👨‍🎓 I started my college career at a community college. When I transferred to the University of Illinois at Urbana-Champaign, I tried a semester in a humanities major, actually the one I would end up graduating with, and decided I wanted more of a challenge. I chose Electrical Engineering because I had heard it was the hardest major on campus, and that turned out to be true enough to switch back <em>out</em> of the major after a semester, but not before getting a B in ECE 190, where I would learn how circuits built up to logic boards and how to program in LC-3 assembly and a bit of C. This would actually help my intuition a lot later on. And for what it's worth, I didn't consider going into computer science because it was for "smart people."</p> </li> <li> <p>📚 During a summer break in college after I had broken up with my first girlfriend, I decided I needed some time for my soul to heal. I thought reading some Great Literature™ would be a good way to do that, so I picked up a few books by someone I had heard of. As it turns out, I somehow had a brain fart and mixed up the American essayist David Sedaris with the marketer Seth Godin and ended up reading a lot of marketing books, instead. But by that time I had forgotten why I wanted the books in the first place, so I was happy to read, anyway. This led me to the very repetitive world of business books and, by extension, repetitive business ideas.</p> </li> </ul> <p>Yet, even after all of this, I didn't really consider making tech a career until after college. I was more interested in get rich quick and 4 Hour Workweek schemes. After graduating from college, I spent a few months chasing these fruitless ideas until I found an article called "<a rel="external" href="https://web.archive.org/web/20190701201856/https://lifehacker.com/how-to-build-a-web-application-from-scratch-with-no-exp-5336113">How to Build a Web Application from Scratch with No Experience</a>."</p> <p><a rel="external" href="https://twitter.com/briankung/status/780814870392532992">https://twitter.com/briankung/status/780814870392532992</a></p> <p>I was excited to build a web application, run a startup, and learn career skills along the way. There was good alignment. Ironically I still haven't really built any of the web applications I've wanted, personally, but it got me looking into Ruby and Ruby on Rails.</p> <p>At this point, I was definitely 80% interested in startups and 20% interested in programming. But there was some foreshadowing that those ratios might flip. One of the earlier schemes I had come up with was to write books and self publish them. Except that instead of writing, I spent most of the time coming up with a <em>sick website</em> that looked like a literal bookshelf to sell my nonexistent books from. Did the ecommerce work? No, and what the hell is ecommerce? But the website was <em>cool</em> and that's what mattered.</p> <p>After some self-study, I got an internship at the Chicago Teachers Pension Fund (CTPF) as an intern. I had a lot of chutzpah, because I was emailing people to ask if I could be an intern even if there were no listings for internships. My go-to line was "I'm not the senior software engineer you're looking for, but..." and then I'd go into my pitch about how motivated I was. Eventually it worked. Honestly, I wasn't given a lot of guidance - and in fact there was none to give, since CTPF wasn't a Rails shop - but I felt like I was on the way, and I'm grateful for the opportunity.</p> <p>Nearly simultaneously, I found out about a coding bootcamp called Code Academy in Chicago which would eventually rename itself Starter League. I followed a lot of their students, who were highly recommended to write blog posts, and got very hyped up about the entire "code as literacy" movement and bootcamps in general, so I was very excited to apply. They gave me a short window of time to accept and I had to crowdfund around $3000 in order to attend. I have a lot of my friends and family to thank for supporting me, though it would weigh on me later. I hung up my hat at the CTPF and went to attend the bootcamp.</p> <p>It was less like the bootcamps of today, which are full-time, and more like an paid club, since it was at most something like 6 hours of instruction per week, but I really liked the people and the environment. We operated out of the newly constructed 1871, a coworking space in the Chicago Merchandise Mart. After being one of the star students, mostly because I had done a good amount of self-study beforehand, I realized I was helping people out more than I was being helped. I was also fielding questions from other people in my life and the greater Chicago ecosystem asking about my impressions of the course and I realized...<a rel="external" href="https://web.archive.org/web/20130328112015/http://www.callmekung.com/2013/03/i-cant-recommend-attending-starter-league/">I just couldn't recommend it.</a></p> <p>I get a high off of buying things when they're on sale - I don't do it often, but if the discount is large enough, it's hard for me to resist. It's part of my immigrant background. I'm used to saving and scrounging, whether that's hoarding takeout containers and plastic bags, or buying in bulk when there are flash sales on groceries. Code Academy was not a good deal, and I had to come clean about it.</p> <p>On the other hand, I was really drinking the kool-aid when it came to startup hustle culture. There's no doubt that that kind of energy and dedication helped me get into the profession, even though I've renounced it since then. I was applying everywhere, meeting tons of people, and overall ⚡️ <em>killing it, bro</em> ⚡️.</p> <p>👉 Fun side story: I showed up to an interview in shorts and flip flops. I also bombed the technical interview, which had a lot of CSS questions, which I shrugged at and said, "I would have to look that up" multiple times. You have to understand, I have almost zero test anxiety, so showing up to interviews half assed is a full ass for me. They very earnestly told me that maybe I should take these things more seriously, wear better clothes, and come more prepared. I very solemnly agreed to their faces and then changed absolutely nothing about myself.</p> <p>Some of the most fun I had was just hanging out and with the troublemakers Jared Steffes, Agam Patel, and Chris Ingebrigtsen on Matador, an investment advice startup that was all sweat equity, as far as I could tell - I don't think anyone was being paid a salary. I was technically "working" for Jared, but really the draw was just being part of a gang of jokesters. I got to put it on my resume and build out some image upload functionality. One time I actually needed the money and asked Jared to cut me a check and it looked like he was drawing the money from his arteries. I don't remember what I needed the money for, but I remember that I was very amused at his pained expression.</p> <p>Still, I consider my first real step on the path to being a software engineer to be my job at Aggrego, which was a digital "skunkworks" embedded in the Chicago Sun-Times which was funded by a madman. The editorial staff hated us because we were well-funded and an annoying distraction, and the plan of throwing money at the paper until it became a successful digital property didn't really pan out, but I did meet some great people like Nikola Ranguelov, Erick Arias, Shannon Carey, and Dave Willkomm, who was a great mentor to me.</p> <p>Aggrego was the most "strict" Agile workplace I've been at since, with velocity tracking, team pointing, burndown charts, retrospectives, product managers, visual and UX designers, and more. It was a great learning experience. There were some downsides, like the madman in charge who full-out screamed at someone with the door open while I was pair programming with an intern - and no, that intern did not want to stay with us. I myself was screamed at when I forgot to show up for a midnight deploy. But I also got to feed my friends takeout when the company agreed to host <a rel="external" href="https://www.eulersprint.org/">Project Euler Sprint</a>, I got a ridiculously high salary for an entry level job at the time ($75,000 in 2012). I almost bought a house!</p> <p>And then of course I quit the moment I got bored.†</p> <p>† ...so I could pursue my own startup.</p> 2022-08-18T21:56:54+00:00 2022-08-18T21:56:54+00:00 Unknown https://briankung.dev/2022/08/18/contributing-to-the-vector-remap-language/ <p>I've been looking for a job doing Rust development and one of the places I applied to was Vector, a DataDog acquisition in the observability space. Observability is one of those terms I had yet to encounter seriously before starting this search, so for the uninitiated:</p> <blockquote> <p>...observability is the ability to measure a system’s current state based on the data it generates, such as logs, metrics, and traces.</p> <p><a rel="external" href="https://www.dynatrace.com/news/blog/what-is-observability-2/">- What is observability?</a> | Dynatrace</p> </blockquote> <p>Now, I didn't get hired at Vector, but I did discover their product, which is an open source, programmable / configurable data pipeline tool. I'll leave the explanation to them:</p> <blockquote> <p>Vector is a high-performance observability data pipeline that enables you to collect, transform, and route all of your logs and metrics.</p> <p>- <a rel="external" href="https://vector.dev/docs/about/what-is-vector/">Vector documentation</a></p> </blockquote> <p>The <a rel="external" href="https://github.com/vectordotdev/vector/">vectordotdev/vector</a> repo was several hundred thousand lines of Rust code. This was probably slightly more complex than my 100 loc CLI app. I was intrigued - would Rust's type system hinder me? Would the compile times grind down my productivity? I wanted to see what it was like to do development on a real production application and get some additional Rust development under my belt, for personal and professional reasons. Thankfully, Vector being open source, I could do that.</p> <hr /> <p>Regarding contributing to open source: as someone who regularly finds employment in the tech industry, and is married to a working spouse with health insurance, I have a certain degree of privilege that affords me the time to contribute to open source. It's important to acknowledge that open source is not a meritocracy and lack of open source contributions should not be considered a flaw in a candidate's application. Working mothers, new entrants to the field, and people from under-represented groups have statistically less free time to make these kinds of contributions. Furthermore, as Ashe Dryden states in <a rel="external" href="https://www.ashedryden.com/blog/the-ethics-of-unpaid-labor-and-the-oss-community#apprenticeships">"The Ethics of Unpaid Labor and the OSS Community"</a>, there are <a rel="external" href="https://www.ashedryden.com/blog/the-ethics-of-unpaid-labor-and-the-oss-community#apprenticeships">a few ways</a> we can work to level the playing field for under-represented groups.</p> <hr /> <p>I got in touch with <a rel="external" href="https://github.com/jszwedko">Jesse</a> and <a rel="external" href="https://github.com/fuchsnj">Nathan</a> on the <a rel="external" href="https://chat.vector.dev/">Vector Discord channel</a> via a former coworker (thanks Nathan Prime!). Jesse was kind enough to tell me about the company and the observability space over Zoom as well as point me to a few good first issues I could tackle on the vector repo and Nathan reviewed my code several times over the course of the month.</p> <p>I decided to tackle <a rel="external" href="https://github.com/vectordotdev/vector/issues/13329">issue #13329</a>, adding a <code>chunk</code> method to the Vector Remap Language (VRL) stdlib.</p> <p>Vector is essentially an ETL (Extract-Transform-Load) platform for shuffling data from one place to another (or in Vector lingo, from Sources to Sinks), and VRL is the domain language for transforming the data. If, for instance, you need to split, parse, or otherwise massage your logs before sending them to your data warehouse, VRL is the language a Vector user would do it in.</p> <p>The rationale for the <code>chunks</code> method was that a Vector user had to send the transformed data to an API with a limitation of 1MB and wanted to be able to split the data into 1MB chunks.</p> <p>I read a great article by the founder of HashiCorp, Mitchell Hashimoto, called "<a rel="external" href="https://mitchellh.com/writing/contributing-to-complex-projects">Contributing to Complex Projects</a>" which was very insightful. The key ideas for me were to "become a user of the project" and "learn down, trace up." The first is self explanatory. The second one was about tracing the execution path of a command down to the leaf nodes and then tracing the operations back up until you understand the entire path.</p> <p>I highly recommend reading Hashimoto's article, but not for the purposes of this blog post because I unfortunately disregarded its advice. I just dove right in and tried to grep my way out. I won't elaborate that much on the journey, but little things like knowing that working on VRL means implementing a <em>programming language</em> might have helped me grok things a bit faster.</p> <p>Adding the <code>chunks()</code> method is basically adding a global method to the standard library that Vector users can use in their VRL scripts. We can separate my pull request to the VRL stdlib into a few different pieces:</p> <ol> <li> <p>Adding the method to the default features. I needed to do this first in order to get the unit tests to actually run:</p> <ul> <li><a rel="external" href="https://github.com/vectordotdev/vector/pull/13794/files#diff-80267028c9325bf0206270a8f0a4ff32b3e97cd544e847bfaa3d5dc9840e994f">lib/vrl/stdlib/Cargo.toml</a></li> <li><a rel="external" href="https://github.com/vectordotdev/vector/pull/13794/files#diff-0e4a5b858d4f321b6db75f693291423714011fbfb6fab0a3fe1967f1f71fbe66">lib/vrl/stdlib/src/lib.rs</a></li> </ul> </li> <li> <p>Implementing the function itself:</p> <ul> <li><a rel="external" href="https://github.com/vectordotdev/vector/pull/13794/files#diff-2166716f04399fa4177d788683336e029a5e889207341eca092761ff5a585e1dR5-R22">fn chunks</a> and <a rel="external" href="https://github.com/vectordotdev/vector/pull/13794/files#diff-2166716f04399fa4177d788683336e029a5e889207341eca092761ff5a585e1dR121-R144">associated unit tests</a></li> </ul> </li> <li> <p>Implementing the function metadata and various checks:</p> <ul> <li> <p>The VRL function's <a rel="external" href="https://github.com/vectordotdev/vector/pull/13794/files#diff-2166716f04399fa4177d788683336e029a5e889207341eca092761ff5a585e1dR32-R45">parameters</a></p> </li> <li> <p><a rel="external" href="https://github.com/vectordotdev/vector/pull/13794/files#diff-2166716f04399fa4177d788683336e029a5e889207341eca092761ff5a585e1dR62-R97">VRL compile time errors</a> and its <a rel="external" href="http://%5Blib/vrl/tests/tests/functions/chunks_chunk_size_at_least_1.vrl%5D(https://github.com/vectordotdev/vector/pull/13794/files#diff-c06e11439418bd09b6bca3fd53b84bd43d52c6ef166a4471402675deb001c84b)">associated test</a></p> </li> <li> <p><a rel="external" href="https://github.com/vectordotdev/vector/pull/13794/files#diff-2166716f04399fa4177d788683336e029a5e889207341eca092761ff5a585e1dR113-R117">VRL fallibility constraints</a> and its <a rel="external" href="http://%5Blib/vrl/tests/tests/functions/chunks_fallible_given_expression.vrl%5D(https://github.com/vectordotdev/vector/pull/13794/files#diff-40d2174476a8fb7e0213860b226bd007f3f1f7028ae682374abfb0fdf173d536)">associated test</a> - more on fallibility later</p> </li> </ul> </li> <li> <p>Finally, documentation:</p> <ul> <li><a rel="external" href="https://github.com/vectordotdev/vector/pull/13794/files#diff-2166716f04399fa4177d788683336e029a5e889207341eca092761ff5a585e1dR47-R60">Examples</a>, which are essentially doctests</li> <li><a rel="external" href="https://github.com/vectordotdev/vector/pull/13794/files#diff-2168de40756007ecadeec59ccc750348cae2db3ca66d061c7dcbf3ce791e15cb">chunks.cue</a>, which is a <a rel="external" href="https://cuelang.org/">CUE lang</a> file for specifying Vector's gorgeous documentation</li> </ul> </li> </ol> <p>It all came together a lot faster when Nathan <a rel="external" href="https://github.com/vectordotdev/vector/pull/13794#discussion_r940608620">suggested that I try out the REPL</a> with <code>cargo run vrl</code>, then I could try to use the function in the terminal. It was a lot easier to understand - hence Hashimoto's advice to become a user of the project. Just goes to show that you can lead a horse to water, but you can't make it drink.</p> <p>(Drinking is following directions. I'm the horse.)</p> <p>Now, one of the reasons VRL is neat is because it makes fallibility a first class concept. If a function is determined to be fallible at compile time, you have to handle the error, much like <a rel="external" href="https://doc.rust-lang.org/stable/nightly-rustc/rustc_lint/unused/static.UNUSED_MUST_USE.html">unused_must_use</a> in Rust. Somewhat confusingly, functions are not inherently fallible or infallible. The determination is made during the compilation process and depends on the inputs.</p> <p>So for <code>chunks()</code>, which takes two parameters, a string and a <code>chunk_size</code> in bytes, we decided to make it infallible if the <code>chunk_size</code> parameter was a literal integer, and fallible otherwise, for instance it it was a method call.</p> <p>So if you have a vrl program chunking a string into single byte chunks like so:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>result = chunks(&quot;abcd&quot;, 1)</span></span> <span class="giallo-l"><span>[&quot;a&quot;, &quot;b&quot;, &quot;c&quot;, &quot;d&quot;]</span></span></code></pre> <p>This is infallible. The <code>chunk_size</code> of <code>1</code> is a literal integer, so the compilation process does not force the user to handle any errors.</p> <p>Meanwhile, if you want a dynamic <code>chunk_size</code>, you get an error:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>result = chunks(&quot;abcd&quot;, int!(floor(4.1)))</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>error[E103]: unhandled fallible assignment</span></span> <span class="giallo-l"><span> ┌─ :1:10</span></span> <span class="giallo-l"><span> │</span></span> <span class="giallo-l"><span>1 │ result = chunks(&quot;abcd&quot;, int!(floor(4.1)))</span></span> <span class="giallo-l"><span> │ -------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^</span></span> <span class="giallo-l"><span> │ │ │</span></span> <span class="giallo-l"><span> │ │ this expression is fallible</span></span> <span class="giallo-l"><span> │ │ update the expression to be infallible</span></span> <span class="giallo-l"><span> │ or change this to an infallible assignment:</span></span> <span class="giallo-l"><span> │ result, err = chunks(&quot;abcd&quot;, int(floor(4.1)))</span></span> <span class="giallo-l"><span> │</span></span> <span class="giallo-l"><span> = see documentation about error handling at https://errors.vrl.dev/#handling</span></span> <span class="giallo-l"><span> = learn more about error code 103 at https://errors.vrl.dev/103</span></span> <span class="giallo-l"><span> = see language documentation at https://vrl.dev</span></span> <span class="giallo-l"><span> = try your code in the VRL REPL, learn more at https://vrl.dev/examples</span></span></code></pre> <p>To account for the fallibility, you must handle the error. Like Go, VRL has multiple return values, one for the value and one for the error:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>result, err = chunks(&quot;abcd&quot;, int!(floor(4.1)))</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>&quot;function call error for \&quot;chunks\&quot; at (14:46): function call error for \&quot;int\&quot; at (29:45): expected integer, got float&quot;</span></span></code></pre> <p>There's currently a bit of tension between fallibility and compilation errors. For instance, we almost decided to <a rel="external" href="https://github.com/vectordotdev/vector/pull/13794#discussion_r944875231">make invalid inputs for <code>chunk_size</code> fallible</a>, but went with making them compile time errors, instead. That way the user wouldn't have to handle the fallibility and then additionally handle the error during runtime, the program just wouldn't compile in the first place.</p> <p>From a language ergonomics perspective, while I still enjoyed using Rust, the compile times did become painful. Compiling a debug version of vector from scratch took 7 minutes and 30 seconds. I later switched to using the mold linker and it went down to 6 minutes, but it was still disruptive. Meanwhile, a full release mode compile would take 14 minutes on my Core i9 iMac. Thankfully, incremental compiles were much quicker.</p> <p>I also haven't dug down into the root cause of the issue yet, but rust-analyzer did not do so well in this code base:</p> <figure> <p><a href="/assets/images/image-3.png"><img src="/assets/images/image-3.png" alt="" /></a></p> <figcaption> <p>Not the most useful type hints</p> </figcaption> </figure> <p>Between the unknown types and rust-analyzer spinning up the CPU fan on every change and establishing a file lock on the package cache that would block the compiler, I got off to a rough start.</p> <p>For context, I didn't really <em>get</em> Rust until I had rust-analyzer working. I had tried Rust a few times before without it or an IDE, and I felt like I was boxing the compiler blindfolded. I'd make a move only to be hammered with a novel type of compiler error with no real way to resolve it. Rust-analyzer's in-editor hints and tight feedback loop really helped me understand the language. So rust-analyzer being less useful and even slowing down compilation was a big difference to my usual Rust workflow. I even turned off rust-analyzer for a bit to see if I got better and faster feedback from the compiler. Ultimately, I turned it back on, but it was painful getting used to the slow analysis and compilation speeds.</p> <p>Ultimately, though, I still enjoyed programming in Rust - how can I get this chain of <code>if let</code> statements to look more readable without actually having <a rel="external" href="https://rust-lang.github.io/rfcs/2497-if-let-chains.html">if-let chains</a>? How can I best use the type system to enforce the boundaries on valid inputs? Also the vector test harness, documentation system, and just working on a programming language (with its own type system) were very cool experiences.</p> <p>So in conclusion, I just need a faster computer 😛</p> <p>If this article inspired you to take a look at the Vector repo, Jesse also pointed me in the direction of this issue which is fairly similar to adding <code>chunks()</code> to the stdlib:</p> <p><a rel="external" href="https://github.com/vectordotdev/vector/issues/13104">Add `zip` function to VRL · Issue #13104 · vectordotdev/vector</a></p> <p>It should follow the same broad strokes as <a rel="external" href="https://github.com/vectordotdev/vector/pull/13794">adding <code>chunks()</code></a>. Take a look and happy coding!</p> <h4 id="additional-resources">Additional Resources</h4> <p>In no particularly useful order:</p> <ul> <li><a rel="external" href="https://chat.vector.dev/">Vector Discord Channel</a></li> <li><a rel="external" href="https://vector.dev/community/">Community | Vector</a></li> <li><a rel="external" href="https://github.com/vectordotdev/vector/pull/13794">feat(vrl): Adds `chunks` function by briankung · Pull Request #13794 · vectordotdev/vector</a></li> <li><a rel="external" href="https://github.com/vectordotdev/vector/issues/13329">Add `chunk` function to split up text into multiple pieces by size · Issue #13329 · vectordotdev/vector</a></li> <li><a rel="external" href="https://github.com/vectordotdev/vector/issues/13104">Add `zip` function to VRL · Issue #13104 · vectordotdev/vector</a></li> <li><a rel="external" href="https://d3rwan.github.io/d3rwan-blog/posts/discovering-vector">Discovering Vector (2020)</a></li> <li><a rel="external" href="https://vector.dev/blog/vector-remap-language/">Vector Remap Language | Vector</a></li> <li><a rel="external" href="https://www.ashedryden.com/blog/the-ethics-of-unpaid-labor-and-the-oss-community#apprenticeships"></a><a rel="external" href="https://www.ashedryden.com/blog/the-ethics-of-unpaid-labor-and-the-oss-community#apprenticeships">The Ethics of Unpaid Labor and the OSS Community | ashe dryden</a></li> </ul> 2022-07-10T12:41:02+00:00 2022-07-10T12:41:02+00:00 Unknown https://briankung.dev/2022/07/10/oxidization-nested-hashes/ <p>This might be an idiosyncrasy of mine, but I tend to use <a rel="external" href="https://ruby-doc.org/core/Hash.html">hashes</a> a lot in Ruby. For instance, recently I was working on a code challenge involving ordering food. There were different categories of courses (breakfast, lunch, and dinner) and different categories of foods (main dish, side dish, etc.), and you need both in order to resolve the final dish name (e.g. <code>lunch + side dish = "banchan"</code>).</p> <p>In Ruby, I would have just used nested hashes. Something like this:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span>menu</span><span style="color: #179299;"> =</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #D20F39;"> breakfast</span><span style="color: #7C7F93;">: { ... },</span></span> <span class="giallo-l"><span style="color: #D20F39;"> lunch</span><span style="color: #7C7F93;">: { ... },</span></span> <span class="giallo-l"><span style="color: #D20F39;"> dinner</span><span style="color: #7C7F93;">: {</span></span> <span class="giallo-l"><span style="color: #D20F39;"> main_dish</span><span style="color: #7C7F93;">:</span><span style="color: #40A02B;"> &quot;bulgogi&quot;</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #D20F39;"> side_dish</span><span style="color: #7C7F93;">:</span><span style="color: #40A02B;"> &quot;banchan&quot;</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #D20F39;"> drink</span><span style="color: #7C7F93;">:</span><span style="color: #40A02B;"> &quot;boricha&quot;</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> }</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span></code></pre> <p>...and I would use it like this:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"># What&#39;s for dinner? 🤔</span></span> <span class="giallo-l"><span>menu</span><span style="color: #7C7F93;">[:</span><span style="color: #D20F39;">dinner</span><span style="color: #7C7F93;">][:</span><span style="color: #D20F39;">main</span><span style="color: #7C7F93;">]</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">#=&gt; &quot;bulgogi&quot;</span></span></code></pre> <p>I struggled with implementing the same pattern in Rust using HashMaps and Enums. First, the enums:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;">#</span><span style="color: #7C7F93;">[</span><span style="color: #DF8E1D;font-style: italic;">derive</span><span style="color: #7C7F93;">(</span><span style="color: #DF8E1D;font-style: italic;">Debug</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> Eq</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> PartialEq</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> Hash</span><span style="color: #7C7F93;">)]</span></span> <span class="giallo-l"><span style="color: #8839EF;">pub enum</span><span style="color: #DF8E1D;font-style: italic;"> Course</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Breakfast</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Lunch</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Dinner</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;">#</span><span style="color: #7C7F93;">[</span><span style="color: #DF8E1D;font-style: italic;">derive</span><span style="color: #7C7F93;">(</span><span style="color: #DF8E1D;font-style: italic;">Debug</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> Eq</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> PartialEq</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> Hash</span><span style="color: #7C7F93;">)]</span></span> <span class="giallo-l"><span style="color: #8839EF;">pub enum</span><span style="color: #DF8E1D;font-style: italic;"> Dish</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Main</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Side</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Drink</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span></code></pre> <p>Enums have a few advantages over using Ruby symbols as keys in the hashmap. You get IDE and compiler help using enums, whereas in Ruby if you misspell a symbol like, for instance, <code>:dinnner</code>, the code will continue to run, it'll just explode somewhere down the line as a runtime error. Rust, on the other hand, will complain during compilation if you spell your variant as <code>Course::Dinnner</code>.</p> <p>Next to create the nested hashmap:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #8839EF;">use</span><span style="color: #DF8E1D;"> std</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">collections</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;font-style: italic;">HashMap</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;">let</span><span> menu</span><span style="color: #179299;"> =</span><span style="color: #DF8E1D;font-style: italic;"> HashMap</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">from</span><span style="color: #7C7F93;">([</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> (</span></span> <span class="giallo-l"><span style="color: #DF8E1D;"> Course</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;font-style: italic;">Breakfast</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #DF8E1D;"> HashMap</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">from</span><span style="color: #7C7F93;">([</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> (</span><span style="color: #DF8E1D;">Dish</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;font-style: italic;">Main</span><span style="color: #7C7F93;">,</span><span style="color: #40A02B;"> &quot;米粥&quot;</span><span style="color: #7C7F93;">),</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> (</span><span style="color: #DF8E1D;">Dish</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;font-style: italic;">Side</span><span style="color: #7C7F93;">,</span><span style="color: #40A02B;"> &quot;肉松&quot;</span><span style="color: #7C7F93;">),</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> (</span><span style="color: #DF8E1D;font-style: italic;">Dish</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;font-style: italic;">Drink</span><span style="color: #7C7F93;">,</span><span style="color: #40A02B;"> &quot;菊花茶&quot;</span><span style="color: #7C7F93;">),</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> ]),</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> ),</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"> // ...</span></span> <span class="giallo-l"><span style="color: #7C7F93;">]);</span></span> <span class="giallo-l"><span style="color: #8839EF;">let</span><span> item</span><span style="color: #179299;"> =</span><span> menu</span></span> <span class="giallo-l"><span style="color: #179299;"> .</span><span style="color: #1E66F5;font-style: italic;">get</span><span style="color: #7C7F93;">(</span><span style="color: #179299;">&amp;</span><span style="color: #DF8E1D;">Course</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;font-style: italic;">Breakfast</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #179299;"> .</span><span style="color: #1E66F5;font-style: italic;">unwrap</span><span style="color: #7C7F93;">()</span></span> <span class="giallo-l"><span style="color: #179299;"> .</span><span style="color: #1E66F5;font-style: italic;">get</span><span style="color: #7C7F93;">(</span><span style="color: #179299;">&amp;</span><span style="color: #DF8E1D;">Dish</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;font-style: italic;">Main</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #179299;"> .</span><span style="color: #1E66F5;font-style: italic;">unwrap</span><span style="color: #7C7F93;">();</span></span></code></pre> <p>However, this led to a disagreement with the compiler because <code>item</code> in this case is a reference to a reference. Its type is <code>&amp;&amp;str</code>, which was incompatible with the rest of my function signatures (<code>&amp;str</code>), and I couldn't dereference it for some reason (unfortunately lost to an uncommitted rendition of code). I also wanted <code>menu</code> to be a global static constant that I can <code>use</code> elsewhere, which involved using <code>once_cell</code>, and it was just getting overly complicated.</p> <p>All this led to a rethinking of the problem. I had two keys and wanted to resolve a label for each combination thereof. The HashMap was an implementation detail. Thankfully, I realized Rust had what I need to do resolution of an arbitrary number and type of keys - pattern matching!</p> <p>Instead of complicated nested hashmaps, I could have a function <code>get_item()</code> that is just a function that matches on <code>Course</code> and <code>Dish</code> and returns a <code>&amp;str</code>. Since it's a function, it's easy to import in other modules. Since it uses pattern matching, it enforces matches, so adding different <code>Courses</code> and <code>Dishes</code> (which I did) results in compiler warnings which helped me shore up the code. Using a built in meant no more importing <code>HashMap</code> and no more chains of <code>get()</code> and <code>unwrap()</code>.</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #8839EF;">fn</span><span style="color: #1E66F5;font-style: italic;"> get_item</span><span style="color: #7C7F93;">(</span><span style="color: #E64553;">course</span><span style="color: #179299;">: &amp;</span><span style="color: #DF8E1D;font-style: italic;">Course</span><span style="color: #7C7F93;">,</span><span style="color: #E64553;"> dish</span><span style="color: #179299;">: &amp;</span><span style="color: #DF8E1D;font-style: italic;">Dish</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> -&gt; &amp;</span><span style="color: #7C7F93;">&#39;</span><span style="color: #1E66F5;font-style: italic;">static</span><span style="color: #DF8E1D;font-style: italic;"> str</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #8839EF;"> match</span><span style="color: #7C7F93;"> (</span><span>course</span><span style="color: #7C7F93;">,</span><span> dish</span><span style="color: #7C7F93;">) {</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> (</span><span style="color: #DF8E1D;font-style: italic;">Course</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;font-style: italic;">Breakfast</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> Dish</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;font-style: italic;">Main</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =&gt;</span><span style="color: #40A02B;"> &quot;Oatmeal&quot;</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> (</span><span style="color: #DF8E1D;font-style: italic;">Course</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;font-style: italic;">Lunch</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> Dish</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;font-style: italic;">Main</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =&gt;</span><span style="color: #40A02B;"> &quot;Fried Rice&quot;</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> (</span><span style="color: #DF8E1D;font-style: italic;">Course</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;font-style: italic;">Dinner</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> Dish</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;font-style: italic;">Main</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =&gt;</span><span style="color: #40A02B;"> &quot;Curry&quot;</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"> // ...</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> }</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span> <span class="giallo-l"><span style="color: #8839EF;">let</span><span> item</span><span style="color: #179299;">: &amp;</span><span style="color: #DF8E1D;font-style: italic;">str</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> get_item</span><span style="color: #7C7F93;">(</span><span style="color: #179299;">&amp;</span><span style="color: #DF8E1D;">Course</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;font-style: italic;">Breakfast</span><span style="color: #7C7F93;">,</span><span style="color: #179299;"> &amp;</span><span style="color: #DF8E1D;">Dish</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;font-style: italic;">Main</span><span style="color: #7C7F93;">);</span></span></code></pre> <p>It worked out quite nicely and made me rethink my use of nested hashes, particularly in Rust but also in Ruby, given the recent addition of native pattern matching:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span>menu</span><span style="color: #179299;"> =</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #D20F39;"> dinner</span><span style="color: #7C7F93;">: {</span></span> <span class="giallo-l"><span style="color: #D20F39;"> main</span><span style="color: #7C7F93;">:</span><span style="color: #40A02B;"> &quot;fried rice&quot;</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #D20F39;"> side</span><span style="color: #7C7F93;">:</span><span style="color: #40A02B;"> &quot;green beans&quot;</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #D20F39;"> drink</span><span style="color: #7C7F93;">:</span><span style="color: #40A02B;"> &quot;tea&quot;</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> }</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span> <span class="giallo-l"><span>menu </span><span style="color: #8839EF;">in</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #D20F39;"> dinner</span><span style="color: #7C7F93;">: {</span></span> <span class="giallo-l"><span style="color: #D20F39;"> main</span><span style="color: #7C7F93;">:</span><span> main</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #D20F39;"> side</span><span style="color: #7C7F93;">:</span><span> side</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #D20F39;"> drink</span><span style="color: #7C7F93;">:</span><span> drink</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> }</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;">puts</span><span style="color: #40A02B;"> &quot;Today we will be eating </span><span style="color: #7C7F93;">#{</span><span>main</span><span style="color: #7C7F93;">}</span><span style="color: #40A02B;"> with a side of </span><span style="color: #7C7F93;">#{</span><span>side</span><span style="color: #7C7F93;">}</span><span style="color: #40A02B;"> and </span><span style="color: #7C7F93;">#{</span><span>drink</span><span style="color: #7C7F93;">}</span><span style="color: #40A02B;">.&quot;</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"># run.rb:9: warning: One-line pattern matching is experimental, and the behavior may change in future versions of Ruby!</span></span> <span class="giallo-l"><span style="color: #FE640B;">Today</span><span> we will be eating fried rice with a side of green beans </span><span style="color: #179299;">and</span><span> tea</span></span></code></pre> <p>Neat! One less abuse of nested hashes. Though in Ruby it's noticeably less ergonomic to use pattern matching than it is in Rust, so I may stick with nested hashes for this particular use case in Ruby.</p> <p>You can find all the Rust code for this post in this playground link: <a rel="external" href="https://play.rust-lang.org/?version=stable&amp;mode=debug&amp;edition=2021&amp;gist=03ddd0d98364d7a886decbf08d15ed96">https://play.rust-lang.org/?version=stable&amp;mode=debug&amp;edition=2021&amp;gist=03ddd0d98364d7a886decbf08d15ed96</a></p> 2022-06-14T12:10:11+00:00 2022-06-14T12:10:11+00:00 Unknown https://briankung.dev/2022/06/14/rust-advocacy-at-a-medium-sized-startup/ <p>At Hologram, I loved getting to sign into work Slack and share knowledge with fellow software engineers of all experience levels. There were about 40 engineers in a company closing in on 200 employees.</p> <p>Meanwhile, the biggest change in my career as a software engineer recently has been learning Rust, which I've been calling a "compiler-guided tour to systems programming." For me, it has been that and more - I accidentally fell in love with Rust's well-crafted APIs, its high assurance qualities, and its community of like-minded and diverse Rustaceans.</p> <p>I still kept quiet about it because I thought it was irrelevant to our current stack. I also thought that my job as a software engineer was primarily to write code. I hadn't considered that organizational initiatives were a natural extension of a software engineer's - or more to the point, any employee's - role.</p> <p>That lasted until my coworker Doug unleashed the floodgates by inviting me to pitch Rust in one of the company's weekly "Backend Guild" video calls (thanks, Doug! 👋). I can't remember what I said at the moment, but I tried to quickly and concisely sum up the aspects of Rust that appealed to me. It must have sounded at least somewhat compelling, though, because someone encouraged us to work on a spike and some proof of concepts, to which I responded by impulsively creating a Slack channel dedicated for what would become the Rust spike initiative.</p> <figure> <p><a href="/assets/images/ikj0dtd2q1f70pjtxzejahfajh0lkkcdtnuxmwt8dl0.jpg"><img src="/assets/images/ikj0dtd2q1f70pjtxzejahfajh0lkkcdtnuxmwt8dl0.jpg" alt="A technicolor logo with (I think) 1980s styling proclaiming &quot;Rust Evangelism Strike Force&quot;" /></a></p> <figcaption> <p><em>Rust Evangelism Strike Force, coming soon to a company near you!</em></p> </figcaption> </figure> <p>Hologram had "Flex Fridays" in which no meetings were scheduled and you could work in a more self-directed way. Flex Fridays became a flurry of Rust-based initiatives for me, including onboarding people interested in Rust onto the ecosystem and walking them through some of the highlights, writing proof of concept code, and writing documentation - so much documentation. Everything became documentation - the merge requests (MRs are PRs in GitLab parlance), Backend Guild presentations, and one-on-one screensharing sessions were all captured and labeled in Notion to build what I hoped would become a library of Hologram-specific resources for onboarding on to the Rust ecosystem.</p> <p>Alas, it was not to be. So here are my hopes and dreams, as well as some of the obstacles we met along the way.</p> <p>I will preface the rest of this post by saying that it is not quantitative by any means. I would have loved to show a histogram of lines of Rust code in production or numbers of developers onboarded, but I never got that far before the company laid off 40% of staff, myself included.</p> <h2 id="why-not-go">"Why not Go?"</h2> <p><a href="/assets/images/1-ifpd_htdik9u6h68szgnua.png"><img src="/assets/images/1-ifpd_htdik9u6h68szgnua.png" alt="" /></a></p> <p>The issue of whether or not to use Go instead of Rust seems inevitable in any organization. I personally have not used Go in any professional capacity, so my authority on the topic is limited. I will say that, from what I've heard of it, I would likely be more excited to learn and use Go than, for example, Python.</p> <p>(Sorry Python folks, it's close enough to Ruby to give me an uncanny valley effect and I also don't want to evaluate which of the many dependency management systems that have sprung up will work the best for me.)</p> <p>Here are the good things I've heard about Go:</p> <ul> <li>Straightforward language and syntax</li> <li>Fast compile times</li> <li>"Batteries included" for web development</li> <li>Concurrency primitives included</li> <li>Extremely good performance out of the box</li> <li>Cross-platform compilation</li> <li>Statically typed</li> <li>Compiles down to binaries</li> </ul> <p>I <em>absolutely</em> did not want this to turn into a language flame war, but unfortunately there's something about either the closeness of the release dates of the two languages or their mutual reputations as "systems" programming languages that brings them into contrast with each other. I still don't want to talk about either language in the framework of "Go vs. Rust," so I'll talk about the context that directly lead to that comparison, instead.</p> <p>Shortly before I had joined Hologram, there was an official decision made in the engineering department to adopt two more languages: Go and Typescript. Naturally, when we started talking about adding yet another language, comparisons with Go came up, though in retrospect, it's a bit puzzling why Typescript was largely left out of that conversation.</p> <p>When I started to join Rust advocacy efforts, Doug and I were counseled to know our "opponent" (Go) inside and out, to position Rust as being better than Go, and to really make a business savvy case for Rust. I took this advice to heart, but I also felt that something was off.</p> <p>I didn't want to talk about Go because it's simply not what I was passionate about. I also didn't want to put down Go users in the company. I just wanted to extol the benefits of Rust and see if anyone was interested in trying it out.</p> <p>Which brings me to my next point. I thought at first that I had to convince The Company to add it to The Stack™. In actual fact, I had two broad categories of stakeholders to convince, not one Company.</p> <h2 id="stakeholder-1-the-business">Stakeholder #1: The Business</h2> <p><a href="/assets/images/the-lego-movie-lord-business-1-dragged.jpg"><img src="/assets/images/the-lego-movie-lord-business-1-dragged.jpg" alt="" /></a></p> <p>The Business wants to know why we should invest engineering time into another technology. The Business is about Return on Investment (ROI), and The Business will make do with what we have in order to get things done on time and under budget. The Business will make nuanced arguments about balancing tech debt versus expansion.</p> <p>I'm not disparaging business interests - it's important to understand, and it's part of why any of us are even employed. In fact, one of the biggest lessons I learned from Hologram is how every engineer can focus on <a rel="external" href="http://www.effectiveengineer.com/">leverage</a> in order to have the biggest positive impacts on business.</p> <p>I had a few ideas on how to convince The Business that Rust was worth investing in.</p> <h3 id="bug-case-studies">Bug case studies</h3> <p>I kept an eye out fixes for bugs that would have been entirely prevented or greatly mitigated in a Rust ecosystem. All of the ones I can remember (invalid array accesses, null return value, empty set checks) would have been caught by returning Rust's <code>Option</code> variants for operations.</p> <p>Then I estimated how much time it took each developer to find and fix these bugs. Developer time translates into money, and these types of bugs were costing a pretty penny. There were two particularly bad weeks where using Rust would have saved a combined 3 weeks worth of developer time. Whether or not they caused any customer-facing issues is also worth noting.</p> <p>I made sure to write the case studies up, including what happened, how it impacted engineering time, and how writing such bugs would have been impossible or unidiomatic in Rust.</p> <p>I presented the case studies during a Backend Guild video call and filed the recording away for later reference.</p> <h3 id="the-case-against-go">The case against Go</h3> <p>We were challenged to provide a <em>gap analysis</em> for using Rust over Go. I'll go into more detail about why I didn't like that framing in a separate section, but to sum up in bullet points, I said:</p> <ul> <li>Error handling in Go is optional and not ergonomic</li> <li>Nils and other issues can slip by Go's type system</li> <li>Rust's platform support means that it can be deployed to anywhere from embedded systems to backend web servers</li> </ul> <p>I read a lot about Go experiences, like these:</p> <ul> <li><a rel="external" href="https://typesanitizer.com/blog/go-experience-report.html">Experience Report: 6 months of Go</a></li> <li><a rel="external" href="https://eng.uber.com/data-race-patterns-in-go/">Data Race Patterns in Go</a></li> <li><a rel="external" href="https://fasterthanli.me/articles/lies-we-tell-ourselves-to-keep-using-golang">Lies we tell ourselves to keep using Golang</a></li> </ul> <p>I also made sure to connect Rust's benefits over Go with the Engineering department's started charter values, like how we generally end up going all in for the difficult, more widely applicable technologies, and how we needed to be business savvy - and squashing bugs before they reach customers is business savvy, especially for an infrastructure company like Hologram!</p> <p>Yet, it made me sad to look up articles on the language just to, in effect, bash it. I'm not generally in the business of discouraging people from coding, regardless of what their favorite programming language was. If I were, I don't think I would have been working on a legacy PHP code base, myself.</p> <p>Which brings me to the second category of stakeholders who we needed to convince.</p> <h2 id="stakeholder-2-the-engineers">Stakeholder #2: The Engineers</h2> <figure> <p>[](/assets/images/20101209-1.gif) <figcaption> <p><em>Apologies to <a rel="external" href="https://www.smbc-comics.com/index.php?db=comics&amp;id=2088#comic">Saturday Morning Breakfast Cereal</a> for my hack job</em></p> </figcaption> </figure> <p>In a perfectly rational world, we would all make perfectly rational decisions for ourselves and for the business. These decisions would be carefully weighed until the optimal time and then executed.</p> <p>But that's not how humans work. For one, we all have different priorities. There was the engineering manager who was constantly pushing us to write more documentation. There was devops, who wrote a Go CLI tool to orchestrate pipeline tasks. There was the billing department, which I was on, which put out fires necessary to earning the trust of our customers and their wallets. And there were plenty of engineers who just wanted to go along their days and get things done.</p> <p>Ultimately, it was these folks who we had to convince. In retrospect, while the business had been officially convinced of the use of Go and Typescript through our decision making processes, the proof, as they say, is in the pudding, and engineers were still writing Javascript and PHP. Their minds may have been made, but their hearts hadn't been won.</p> <p>This was also where, in my opinion, the real difference was made. If we won people's hearts, the business case for the language would be made, as all the engineers would want to write Rust, anyway.</p> <p>Here's how I had been planning to win my fellow colleagues over to the dark side:</p> <h3 id="personalized-onboarding">Personalized onboarding</h3> <p>One of the first things I offered to do was to use our Flex Fridays to onboard people to the new language. It entailed getting a working Rust toolchain up and running and maybe a brief tour of high level language features.</p> <p>I tried to remember to ask if we could record each session, both for later publication and to see if there were common aspects of onboarding that were problematic. I viewed it as an exercise in customer development, Lean Startup style, as much as an active tutorial.</p> <p>We did a few of these with people from different backgrounds. One principal engineer with Swift experience compared it to guard lets and other features from Swift. Another engineer found the tooling and ecosystem quite fascinating. And a QA team lead had a lot of fun in our introduction to Rust's built-in testing capabilities.</p> <p><strong>Goals:</strong></p> <ul> <li>Get people over the "getting started" hump</li> <li>Give high level overview of Rust toolchain features</li> <li>Bolster documentation with videos</li> </ul> <p><strong>Metrics:</strong></p> <ul> <li>Number of engineers onboarded: 4</li> <li>Videos added to documentation: 2</li> </ul> <h3 id="proof-of-concepts">Proof of concepts</h3> <p>In our use of the term proof of concept, it was less about feasibility studies and more about small code spikes to disseminate learning. To that end, I wrote one proof of concept myself, a cross-platform (Unixes + Windows!) hosts file editor for local development, and contributed to another proof of concept that Doug had started involving DNS.</p> <p>For both of these, I posted MRs to the Rust slack channel so that everyone could get context. And in the MRs themselves, I went into great detail on the Rust features I was using and why, like <a rel="external" href="https://doc.rust-lang.org/stable/book/ch05-01-defining-structs.html#creating-instances-from-other-instances-with-struct-update-syntax">Struct Update Syntax</a>, or <a rel="external" href="https://doc.rust-lang.org/rust-by-example/meta/doc.html#doc-comments">doc comments</a>. Of course, in a production code base with shared context around the language and ecosystem, these explanations would be overly long winded and superfluous. But since our deliverable for these proof of concepts was education, it was appropriate to bloviate a little.</p> <p><strong>Goals:</strong></p> <ul> <li>Write proof of concepts</li> <li>Submit code changes for review by non-Rust users</li> <li>Assess feasibility of using Rust</li> </ul> <p><strong>Metrics:</strong></p> <ul> <li>Proof of concepts written: 2</li> <li>Merge requests: 6</li> <li>Merge requests with extra details: 3</li> </ul> <h3 id="presentations">Presentations</h3> <p>As I mentioned before, we had a Backend Guild video call every week. I like public speaking, so I used the opportunity to talk about Rust when I had something to talk about.</p> <p>One of the engineering managers mentioned that she needed better documentation, so I whipped up a quick demonstration of Rust doc comments and doc tests. It helps that documentation is a first class citizen in the Rust ecosystem, so it was easy to do so on the fly. It was a demonstration of the built in documentation, how doc comments get included in the by the doc builder, and how the doc tests are executed and run as code.</p> <p>I also presented an argument for Rust, the language as a whole, with the main points being the documentation, safety (in terms of impact of bugs on the business), and community. I think Rust's diverse and inclusive community (and therefore steering parties via teams and work groups) is perhaps one of its biggest strengths, which manifests itself in a very nuanced, safe, and comprehensive language.</p> <p><strong>Goals:</strong></p> <ul> <li>Introduce the general engineering department to Rust and specific Rust features</li> <li>Keep engineering updated on Rust spike initiatives</li> <li>Add videos to documentation</li> </ul> <p><strong>Metrics:</strong></p> <ul> <li>Rust talks given: 3</li> <li>Recordings made: 3</li> </ul> <h3 id="documentation">Documentation</h3> <p>Because communication and education were key to the Rust spike initiative, documentation was key. I wrote a homepage on our Notion documents entitled "Rust Spike 🦀" which housed all of the above:</p> <ul> <li>Simple getting started guides</li> <li>Links to proof of concepts</li> <li>Onboarding and presentation videos</li> </ul> <p>Additionally, I started an <code>#eng_spike_rust</code> Slack channel which we casually posted news and MRs to.</p> <h3 id="on-metrics">On metrics</h3> <p>I would have loved to have better metrics of engagement rather than just results, and I hope that someone takes up the mantle and introduces those metrics in my absence. But in an ideal scenario where Rust is officially added to the stack, we would have been able to track better metrics for Rust uptake (depending on goals):</p> <ul> <li>Proportion of Rust code to legacy code</li> <li>Proportion of new code bases written in Rust</li> <li>Performance and maintenance in Rust vs non-Rust code <ul> <li>Bug rates</li> <li>Uptime</li> <li>Incident rates</li> <li>Performance</li> </ul> </li> <li>Usability <ul> <li>Number of authors of Rust code</li> <li>Time to first contribution</li> <li>Satisfaction surveys</li> </ul> </li> </ul> <p>...and these are just a first pass at what would hopefully be useful metrics. We would likely need to redo based on what ends up being useful to measure.</p> <h2 id="conclusion">Conclusion</h2> <p>I think company-wide goals and metrics were one place where, if the Go initiative at the company had been stronger, there wouldn't have been as much of a vacuum for another language to be considered for general use. Several teams thriving on using Go for their purposes would have made it harder for an arguably marginally better language to shine. Instead, the language stayed siloed in a single repository owned by one team, and the rest of the backend engineers were either stuck on PHP or starting out on Python. In this environment, Rust had a chance, so we made a case for it.</p> <p>The success of a new language at a company is about the uptake by the individual contributors, and that is a campaign that's best made one engineer at a time. That was my starting point, but it would be foolish to leave the business out of the calculus, so I'm grateful to have been pushed to make a business case for the language as well.</p> <p>In the end, my time at the company came to a close before I could see the Rust initiative to any sort of fruition, but I'm grateful that Hologram had that kind of openness to engineer initiatives. It allowed me to learn a lot about advocacy and organizational change.</p> <p>...even so, I hope to land somewhere that already uses Rust for my next job!</p> 2022-06-11T02:46:41+00:00 2022-06-11T02:46:41+00:00 Unknown https://briankung.dev/2022/06/11/php-is-not-for-me/ <p>So most recently I was employed at Hologram and was therefore too busy to write. Not anymore - I was one of the employees laid off from Hologram recently.</p> <p>While at Hologram, I worked mostly in PHP, which was something I actually dreaded coming into the job, but ended up not minding as much as I thought I would. PHP, for all of its terrible reputation, functions mostly like how you'd expect a programming language to function. It does what you ask of it. But there are a lot of dark corners to the language.</p> <h2 id="quick-syntax-oddities">Quick syntax oddities</h2> <p>But there are some...oddities. I'll start with the syntax. As a Rubyist, the opening sigil for declaring or referencing a variable feels <em>super</em> weird:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="php"><span class="giallo-l"><span>$a</span><span style="color: #179299;"> =</span><span style="color: #FE640B;"> 1</span><span style="color: #7C7F93;">;</span></span></code></pre> <p>I made this joke to my team: "PHP must be great for startups, because all I see are dollar signs." Then I proceeded to tell dad jokes for 3 months until management got rid of me. I guess that one was on me.</p> <p>Then there's stabby arrow <a rel="external" href="https://www.php.net/manual/en/language.oop5.properties.php">class property</a> calls, which include functions:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="php"><span class="giallo-l"><span style="color: #8839EF;">class</span><span style="color: #DF8E1D;font-style: italic;"> Dog</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"> // This is a property</span></span> <span class="giallo-l"><span style="color: #8839EF;"> public</span><span> $fur_color</span><span style="color: #179299;"> =</span><span style="color: #40A02B;"> &quot;brown&quot;</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #8839EF;"> function</span><span style="color: #1E66F5;font-style: italic;"> woof</span><span style="color: #7C7F93;">() {</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;"> echo</span><span style="color: #40A02B;"> &quot;woof</span><span style="color: #EA76CB;">\n</span><span style="color: #40A02B;">&quot;</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> }</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>$d</span><span style="color: #179299;"> =</span><span style="color: #8839EF;"> new</span><span style="color: #DF8E1D;font-style: italic;"> Dog</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>$d</span><span style="color: #179299;">-&gt;</span><span>fur_color</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">// &quot;brown&quot;</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>$d</span><span style="color: #179299;">-&gt;</span><span style="color: #1E66F5;font-style: italic;">woof</span><span style="color: #7C7F93;">();</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">// woof</span></span></code></pre> <p>Another oddity is the array push syntax, which looks like an assignment</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="php"><span class="giallo-l"><span>$a</span><span style="color: #179299;"> =</span><span style="color: #7C7F93;"> [];</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">// []</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>$a</span><span style="color: #7C7F93;">[]</span><span style="color: #179299;"> =</span><span style="color: #FE640B;"> 1</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">// [1]</span></span></code></pre> <p>But those are just some quirks! They're relatively easy to get over. Other issues are not.</p> <h2 id="everything-is-a-string">Everything is a string</h2> <p>In PHP, if you want to refer to a function, you can just use a string as if it were the function itself:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="php"><span class="giallo-l"><span style="color: #8839EF;">function</span><span style="color: #1E66F5;font-style: italic;"> say_hi</span><span style="color: #7C7F93;">() {</span><span style="color: #1E66F5;font-style: italic;">echo</span><span style="color: #40A02B;"> &quot;hi</span><span style="color: #EA76CB;">\n</span><span style="color: #40A02B;">&quot;</span><span style="color: #7C7F93;">;}</span></span> <span class="giallo-l"><span style="color: #40A02B;">&#39;say_hi&#39;</span><span style="color: #7C7F93;">();</span></span> <span class="giallo-l"><span>hi</span></span></code></pre> <p>And if you want to reference a class, you can also use a string:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="php"><span class="giallo-l"><span>$klass</span><span style="color: #179299;"> =</span><span style="color: #40A02B;"> &#39;Dog&#39;</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span>$d</span><span style="color: #179299;"> =</span><span style="color: #8839EF;"> new</span><span> $klass</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;">var_dump</span><span style="color: #7C7F93;">(</span><span>$d</span><span style="color: #7C7F93;">);</span></span> <span class="giallo-l"><span style="color: #8839EF;">object</span><span style="color: #7C7F93;">(</span><span>Dog</span><span style="color: #7C7F93;">)</span><span style="color: #7C7F93;font-style: italic;">#2 (0) {</span></span> <span class="giallo-l"><span>}</span></span></code></pre> <p>Oddly enough, using a raw string to reference a class doesn't work:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="php"><span class="giallo-l"><span>$d</span><span style="color: #179299;"> =</span><span> new</span><span style="color: #40A02B;"> &#39;Dog&#39;</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span>PHP Parse error</span><span style="color: #7C7F93;">:</span><span> syntax error</span><span style="color: #7C7F93;">,</span><span> unexpected</span><span style="color: #40A02B;"> &#39;&#39;</span><span>Dog</span><span style="color: #40A02B;">&#39;&#39;</span><span style="color: #7C7F93;"> (</span><span>T_CONSTANT_ENCAPSED_STRING</span><span style="color: #7C7F93;">)</span><span> in php shell code</span><span style="color: #8839EF;"> on</span><span> line</span><span style="color: #FE640B;"> 1</span></span></code></pre> <p>Why does this matter? Well, PHP also really likes to do metaprogramming with string composition. So I encountered a lot of code like this:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="php"><span class="giallo-l"><span>$namespaced_klass</span><span style="color: #179299;"> =</span><span style="color: #40A02B;"> &quot;\Namespace</span><span style="color: #EA76CB;">\\</span><span style="color: #40A02B;">&quot;</span><span style="color: #179299;"> .</span><span> $klass</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>$object</span><span style="color: #179299;"> =</span><span style="color: #8839EF;"> new</span><span> $namespaced_klass</span><span style="color: #7C7F93;">();</span></span></code></pre> <p>The <code>.</code> concatenates the two strings. Let's try this in Ruby:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span>klass</span><span style="color: #179299;"> =</span><span style="color: #40A02B;"> &quot;Animal::&quot;</span><span style="color: #179299;"> +</span><span style="color: #40A02B;"> &quot;Dog&quot;</span></span> <span class="giallo-l"><span style="color: #179299;">=&gt;</span><span style="color: #40A02B;"> &quot;Animal::Dog&quot;</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>d</span><span style="color: #179299;"> =</span><span> klass</span><span style="color: #7C7F93;">.</span><span style="color: #8839EF;">new</span><span style="color: #7C7F93;">()</span></span> <span class="giallo-l"><span style="color: #FE640B;">Traceback</span><span style="color: #7C7F93;"> (</span><span>most recent call last</span><span style="color: #7C7F93;">):</span></span> <span class="giallo-l"><span style="color: #FE640B;"> 4</span><span style="color: #7C7F93;">: ...</span></span> <span class="giallo-l"><span style="color: #FE640B;">NoMethodError</span><span style="color: #7C7F93;"> (</span><span>undefined method </span><span style="color: #40A02B;">`new&#39; for &quot;Animal::Dog&quot;:String)</span></span></code></pre> <p>Both languages make string manipulation really easy, but in Ruby, it's a lot harder to summon a class or a function from a string, which is actually a feature.</p> <p>I would oftentimes come across a function like this in our PHP code base:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="php"><span class="giallo-l"><span style="color: #8839EF;">function</span><span style="color: #1E66F5;font-style: italic;"> setupClass</span><span style="color: #7C7F93;">(</span><span style="color: #E64553;">$</span><span style="color: #E64553;font-style: italic;">klass</span><span style="color: #7C7F93;">) {</span></span> <span class="giallo-l"><span> $klass</span><span style="color: #179299;"> =</span><span style="color: #40A02B;"> &quot;\GlobalNamespace</span><span style="color: #EA76CB;">\\</span><span style="color: #40A02B;">&quot;</span><span style="color: #179299;"> .</span><span> $klass</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span> $thing</span><span style="color: #179299;"> =</span><span style="color: #8839EF;"> new</span><span> $klass</span><span style="color: #7C7F93;">();</span></span> <span class="giallo-l"><span> $thing</span><span style="color: #179299;">-&gt;</span><span style="color: #1E66F5;font-style: italic;">doSomethingIncomprehensible</span><span style="color: #7C7F93;">();</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span></code></pre> <p>...and I would have no idea where the function <code>doSomethingIncomprehensible</code> would come from, which would add layers of indirection. When you follow the references of <code>setupClass</code>, it was possible to find even more indirection:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="php"><span class="giallo-l"><span style="color: #8839EF;">if</span><span style="color: #7C7F93;"> (</span><span>$some_condition</span><span style="color: #7C7F93;">) {</span></span> <span class="giallo-l"><span> $klass</span><span style="color: #179299;"> =</span><span> $a</span><span style="color: #179299;"> .</span><span> $b</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span><span style="color: #8839EF;"> else</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span> $klass</span><span style="color: #179299;"> =</span><span> $a</span><span style="color: #179299;"> .</span><span> $c</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;">setupClass</span><span style="color: #7C7F93;">(</span><span>$klass</span><span style="color: #7C7F93;">);</span></span></code></pre> <p>It's very <em>convenient</em> for metaprogramming, but it encourages some truly convoluted code.</p> <p>Also as we can see with trying to instantiate raw string <code>'Dog'</code> that it's inconsistent.</p> <h2 id="error-messages-are-not-helpful">Error messages are not helpful</h2> <p>I am very much cheating here comparing PHP to Rust, but let's compare what happens when you call a slightly incorrect function name in various languages. Assuming we have a <code>woof</code> method defined on a struct / class and we call <code>woog()</code> instead of <code>woof()</code>:</p> <p>PHP:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>PHP Warning: Uncaught Error: Call to undefined method Dog::woog() in php shell code:1</span></span> <span class="giallo-l"><span>Stack trace:</span></span> <span class="giallo-l"><span>#0 {main}</span></span> <span class="giallo-l"><span> thrown in php shell code on line 1</span></span></code></pre> <p>Ruby:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>Traceback (most recent call last):</span></span> <span class="giallo-l"><span> 4: from /Users/brian/.rbenv/versions/3.0.0/bin/irb:23:in `&lt;main&gt;&#39;</span></span> <span class="giallo-l"><span> 3: from /Users/brian/.rbenv/versions/3.0.0/bin/irb:23:in `load&#39;</span></span> <span class="giallo-l"><span> 2: from /Users/brian/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/irb-1.3.0/exe/irb:11:in `&lt;top (required)&gt;&#39;</span></span> <span class="giallo-l"><span> 1: from (irb):37:in `&lt;main&gt;&#39;</span></span> <span class="giallo-l"><span>NoMethodError (undefined method `woog&#39; for #&lt;Dog:0x00007f8ec213e450&gt;)</span></span> <span class="giallo-l"><span>Did you mean? woof</span></span></code></pre> <p>Rust:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>error[E0599]: no method named `woog` found for struct `Dog` in the current scope</span></span> <span class="giallo-l"><span> --&gt; src/main.rs:12:7</span></span> <span class="giallo-l"><span> |</span></span> <span class="giallo-l"><span>2 | struct Dog;</span></span> <span class="giallo-l"><span> | ----------- method `woog` not found for this</span></span> <span class="giallo-l"><span>...</span></span> <span class="giallo-l"><span>12 | d.woog();</span></span> <span class="giallo-l"><span> | ^^^^ help: there is an associated function with a similar name: `woof`</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>For more information about this error, try `rustc --explain E0599`.</span></span></code></pre> <p>Both Ruby and Rust offer helpful suggestions. PHP does not.</p> <p>And this is a fairly bog standard error message. PHP error messages get pretty cryptic. The one that stumped me the most was this one:</p> <p><a rel="external" href="https://github.com/mockery/mockery/issues/1038">Serialization of mocked objects in PHP 7.4 · Issue #1038 · mockery/mockery</a>:</p> <blockquote> <p><code>Exception : Serialization of 'ReflectionClass' is not allowed</code></p> </blockquote> <p>Which basically boiled down to somewhere, someone used a class that cannot be serialized by mockery, the unit test framework, and PHP absolutely could not tell me which class it was or give me a stack trace.</p> <h2 id="global-functions-many-deprecated">Global functions, many deprecated</h2> <p>At some point, I wanted to write a function that checks whether a string starts with another string, so I used <a rel="external" href="https://www.php.net/manual/en/function.strpos.php"><code>strpos</code></a>, which purports to "Find the position of the first occurrence of a substring in a string." Which it does, but PHP's type system allows <code>strpos</code> to return either a numeric index or a boolean <code>false</code>.</p> <p>I assumed <code>strpos</code> returned a numeric value for the index, i.e. a string could safely be assumed to start with another string if <code>index == 0</code>. Sadly, PHP casts <code>false</code> to <code>0</code>, a common aspect of a lot of languages, actually, but one that stumped me for a few days as every string seemed to report starting with every other string. No, <code>strpos</code> was just returning <code>false</code>, which was getting cast to <code>0</code>.</p> <p>And yes, I could have double checked with <code>strpos($haystack, $needle) === 0</code> instead of <code>strpos($haystack, $needle) == 0</code>, but I had no idea that <code>strpos</code> could return a boolean.</p> <p>PHP 8, thankfully, has <code>[str_starts_with](https://www.php.net/manual/en/function.str-starts-with.php)</code> and <code>[str_ends_with](https://www.php.net/manual/en/function.str-ends-with.php)</code>, but we were stuck on an earlier version of PHP.</p> <h2 id="few-functional-affordances-cry">Few functional affordances 😢</h2> <p>Ruby has postfix functional methods:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span style="color: #7C7F93;">[</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">2</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">3</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">4</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">5</span><span style="color: #7C7F93;">].</span><span style="color: #1E66F5;font-style: italic;">map</span><span style="color: #7C7F93;"> {|</span><span>num</span><span style="color: #7C7F93;">|</span><span> num</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">pow</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">2</span><span style="color: #7C7F93;">)}</span></span> <span class="giallo-l"><span style="color: #179299;">=&gt;</span><span style="color: #7C7F93;"> [</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;"> 4</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;"> 9</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;"> 16</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;"> 25</span><span style="color: #7C7F93;">]</span></span></code></pre> <p>Which means that you can chain functions that return Enumerables together to stack different effects onto the original array / Enumerable:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span style="color: #7C7F93;">[</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">2</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">3</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">4</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">5</span><span style="color: #7C7F93;">].</span><span style="color: #1E66F5;font-style: italic;">map</span><span style="color: #7C7F93;"> {|</span><span>num</span><span style="color: #7C7F93;">|</span><span> num</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">pow</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">2</span><span style="color: #7C7F93;">)}.</span><span style="color: #1E66F5;font-style: italic;">map</span><span style="color: #7C7F93;"> {|</span><span>num</span><span style="color: #7C7F93;">|</span><span> num </span><span style="color: #179299;">/</span><span style="color: #FE640B;"> 2.0</span><span style="color: #7C7F93;">}</span></span> <span class="giallo-l"><span style="color: #179299;">=&gt;</span><span style="color: #7C7F93;"> [</span><span style="color: #FE640B;">0.5</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;"> 2.0</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;"> 4.5</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;"> 8.0</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;"> 12.5</span><span style="color: #7C7F93;">]</span></span></code></pre> <p>PHP does have a map function, but it's actually a global function:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="php"><span class="giallo-l"><span style="color: #1E66F5;font-style: italic;">array_map</span><span style="color: #7C7F93;">(</span><span style="color: #8839EF;">function</span><span style="color: #7C7F93;">(</span><span style="color: #E64553;">$</span><span style="color: #E64553;font-style: italic;">num</span><span style="color: #7C7F93;">) {</span><span style="color: #8839EF;">return</span><span> $num</span><span style="color: #179299;"> *</span><span> $num</span><span style="color: #7C7F93;">;}, [</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">2</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">3</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">4</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">5</span><span style="color: #7C7F93;">]);</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;">array</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">5</span><span style="color: #7C7F93;">) {</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> [</span><span style="color: #FE640B;">0</span><span style="color: #7C7F93;">]</span><span style="color: #179299;">=&gt;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> int</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> [</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">]</span><span style="color: #179299;">=&gt;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> int</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">4</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> [</span><span style="color: #FE640B;">2</span><span style="color: #7C7F93;">]</span><span style="color: #179299;">=&gt;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> int</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">9</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> [</span><span style="color: #FE640B;">3</span><span style="color: #7C7F93;">]</span><span style="color: #179299;">=&gt;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> int</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">16</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> [</span><span style="color: #FE640B;">4</span><span style="color: #7C7F93;">]</span><span style="color: #179299;">=&gt;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> int</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">25</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span></code></pre> <p>Now suddenly when you want to combine functions it gets very messy, and the order of execution is the opposite of the order you read it in:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="php"><span class="giallo-l"><span style="color: #1E66F5;font-style: italic;">array_map</span><span style="color: #7C7F93;">(</span><span style="color: #8839EF;">function</span><span style="color: #7C7F93;"> (</span><span style="color: #E64553;">$</span><span style="color: #E64553;font-style: italic;">num</span><span style="color: #7C7F93;">) {</span><span style="color: #8839EF;">return</span><span> $num</span><span style="color: #179299;"> /</span><span style="color: #FE640B;"> 2</span><span style="color: #7C7F93;">.</span><span style="color: #FE640B;">0</span><span style="color: #7C7F93;">;},</span><span style="color: #1E66F5;font-style: italic;"> array_map</span><span style="color: #7C7F93;">(</span><span style="color: #8839EF;">function</span><span style="color: #7C7F93;">(</span><span style="color: #E64553;">$</span><span style="color: #E64553;font-style: italic;">num</span><span style="color: #7C7F93;">) {</span><span style="color: #8839EF;">return</span><span> $num</span><span style="color: #179299;"> *</span><span> $num</span><span style="color: #7C7F93;">;}, [</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">2</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">3</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">4</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">5</span><span style="color: #7C7F93;">]));</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;">array</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">5</span><span style="color: #7C7F93;">) {</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> [</span><span style="color: #FE640B;">0</span><span style="color: #7C7F93;">]</span><span style="color: #179299;">=&gt;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> float</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">0</span><span style="color: #7C7F93;">.</span><span style="color: #FE640B;">5</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> [</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">]</span><span style="color: #179299;">=&gt;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> float</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">2</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> [</span><span style="color: #FE640B;">2</span><span style="color: #7C7F93;">]</span><span style="color: #179299;">=&gt;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> float</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">4</span><span style="color: #7C7F93;">.</span><span style="color: #FE640B;">5</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> [</span><span style="color: #FE640B;">3</span><span style="color: #7C7F93;">]</span><span style="color: #179299;">=&gt;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> float</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">8</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> [</span><span style="color: #FE640B;">4</span><span style="color: #7C7F93;">]</span><span style="color: #179299;">=&gt;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> float</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">12</span><span style="color: #7C7F93;">.</span><span style="color: #FE640B;">5</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span></code></pre><h2 id="a-final-petty-complaint">A final petty complaint</h2> <p>All of the above examples were generated in the PHP REPL (<code>php -a</code>) using the <code>[var_dump](https://www.php.net/manual/en/function.var-dump.php)</code> method. Basically, where Ruby's REPL (<code>irb</code>) would immediately print a string representation of the result of the expression, PHP would silently perform (or not perform) the expected operation.</p> <p>Ruby:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span style="color: #1E66F5;font-style: italic;">irb</span><span style="color: #7C7F93;">(</span><span>main</span><span style="color: #7C7F93;">):</span><span style="color: #FE640B;">044</span><span style="color: #7C7F93;">:</span><span style="color: #FE640B;">0</span><span style="color: #179299;">&gt;</span><span style="color: #FE640B;"> 1</span><span style="color: #179299;"> +</span><span style="color: #FE640B;"> 1</span></span> <span class="giallo-l"><span style="color: #179299;">=&gt;</span><span style="color: #FE640B;"> 2</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;">irb</span><span style="color: #7C7F93;">(</span><span>main</span><span style="color: #7C7F93;">):</span><span style="color: #FE640B;">045</span><span style="color: #7C7F93;">:</span><span style="color: #FE640B;">0</span><span style="color: #179299;">&gt;</span></span></code></pre> <p>PHP:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>php &gt; 1 + 1;</span></span> <span class="giallo-l"><span>php &gt;</span></span></code></pre> <p>Unless you <code>var_dump</code> it:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>php &gt; var_dump(1 + 1);</span></span> <span class="giallo-l"><span>int(2)</span></span> <span class="giallo-l"><span>php &gt;</span></span></code></pre> <p>And if you don't, the REPL remains unchanged, which looks frustratingly similar to if you forget to add a semicolon to the end of an expression:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>php &gt; 1 + 1</span></span> <span class="giallo-l"><span>php &gt; echo &quot;hi&quot;;</span></span> <span class="giallo-l"><span>PHP Parse error: syntax error, unexpected &#39;echo&#39; (T_ECHO) in php shell code on line 2</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>Parse error: syntax error, unexpected &#39;echo&#39; (T_ECHO) in php shell code on line 2</span></span> <span class="giallo-l"><span>php &gt;</span></span></code></pre> <p>Note: this error actually means "you forgot a semicolon." See what I mean about the error messages?</p> <p>Now that I think of it, there's probably a command line flag you can pass to <code>php</code> to get it to behave the way I want, but it should be the default.</p> <hr /> <p>Despite all the quirks, it wasn't much harder to become productive in PHP than in other dynamic languages. It just happens to be full of papercuts. And, that said, the language itself is still changing and improving - PHP 8 added enums and pattern matching, for example! I was eager to get my grubby little hands on those, but it didn't happen before my tenure at the company ended.</p> <p>Overall, PHP doesn't <em>spark joy</em> in me, but I respect that it's a beloved tool for many around the world and a workhorse of a language.</p> 2022-04-13T00:15:28+00:00 2022-04-13T00:15:28+00:00 Unknown https://briankung.dev/2022/04/13/how-and-why-our-kids-are-learning-chinese/ <p>"Are you teaching your kids Chinese?" and "how are you teaching your kids Chinese?" are two questions we get a lot from Chinese American parents. One question I wish more parents asked us is "why?"</p> <p>When I was picking a major course of study in college, I consistently gravitated towards East Asian Languages and Culture (EALC) with a focus on Chinese for two reasons that I suspect are similar for our friends with a similar background. Those reasons were:</p> <ol> <li> <p>Cultural proximity</p> </li> <li> <p>Economic anxiety</p> </li> </ol> <h2 id="cultural-proximity">Cultural proximity</h2> <p>China is not a monolithic culture. There are 54-odd minorities recognized by the Chinese government and no doubt there are countless more regional dialects.</p> <p>The variations between dialects of Chinese can be greater than the variation between European languages. For instance, the lexical similarity of Italian and French is 0.89 (out of 1.00 being the same language). Technically, that makes them dialects of each other since the lexical similarity is greater than 0.85 [<a rel="external" href="https://en.wikipedia.org/wiki/Lexical_similarity#Indo-European_languages">Wikipedia</a>]. The lexical similarity between Mandarin and Cantonese Chinese is less well-studied, however some claim it to be less than 0.5 [<a rel="external" href="https://improvemandarin.com/mandarin-vs-cantonese/">source</a>]. Informally, it seems to be true - discounting written language, they're largely mutually unintelligible.</p> <p>All this to say, when I went to college, I didn't have the opportunity to learn either of my heritage languages - Cantonese or Taishanese (Toisan). I got to pick Mandarin Chinese, Japanese, or Korean. So I tried to pick all of them.</p> <p>(I did, but I ended up sticking mostly to Chinese.)</p> <p>Mandarin Chinese was the closest in lexical and cultural similarity, and little did I know that both of my grandmothers spoke it (albeit with colloquial accents).</p> <h2 id="economic-anxiety">Economic anxiety</h2> <p>This one was much more vague, though I suspect it's a much more widely shared reason for learning Chinese than cultural proximity.</p> <p>In the USA, we hear about China as a vague, foreboding kingdom where freedom-hating serfs bitterly manufacture our mass-market goods while plotting to overthrow our global peacekeeping efforts and supplant themselves as the number one world superpower. So the only way to defeat China is to...make a lot of money off of China?</p> <p>The kind of discourse one hears about China in America is unhinged, frankly speaking, but it pervades almost every American's perspective of Chinese language and culture, including Asian and Chinese Americans.</p> <p>So when I was in college, my naive assumption was that learning Chinese would help alleviate economic anxiety. It's hard to write about this because I didn't have a clear idea of what that meant. Similar to how we were told in middle school to learn Spanish to be able to keep up with the business world, I didn't know what it meant for my career to learn Chinese.</p> <p>I <em>hoped</em> that it would mean that either I would have different job opportunities or higher paid job opportunities.</p> <h3 id="different-job-opportunities">Different job opportunities</h3> <p>As I neared graduation, I started to think about what I could do with my degree in EALC. Taking cues from my seniors, they got into:</p> <ul> <li> <p>Horseback riding</p> </li> <li> <p>Pharmaceutical liaison</p> </li> <li> <p>Restaurateur</p> </li> <li> <p>Grad school</p> </li> </ul> <p>...so in short, I learned absolutely nothing. I suspected that the ideal job for an EALC major was to become a translator, ideally for something like the United Nations.</p> <p>Little did I know:</p> <p><a rel="external" href="https://twitter.com/filmwonk/status/1458239012984868867">https://twitter.com/filmwonk/status/1458239012984868867</a></p> <p>This is backed up by my wife's experiences as a medical translator. Cici had a very specialized part time job that required both a doctorate level education in a medical field and high-level understanding of Chinese. There was still no way to make a living doing it.</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>📝 Cici&#39;s note: Medical translation was paid per word, and the more you translated or reviewed per unit time, the more money you made. The rate for this job was $50 USD per 1,000 source characters (when translating Chinese to English). I had to negotiate to get that rate. It took roughly 1 hour per 1000 words, and that&#39;s with looking up similar publications to get a feel for phrasing instead of doing word-for-word translations. Note that this is a freelancing job - these numbers are all before taxes and health insurance. The good part is you get to choose your hours.</span></span></code></pre> <p>Theoretically I could find a job <em>in China</em>, but I would have a significant language disadvantage compared to other job seekers. Furthermore, the kinds of jobs you can find in China aren't necessarily more economically secure than the ones a college graduate could find in America.</p> <p>The competitive landscape is also very different for English speakers now than it was 20 years ago. Many more people around the globe are now fluent in English. More people are learning Chinese in countries with closer cultural proximity, as well.</p> <p>Speaking of which, it's not just linguistic competency, but cultural competency that leads to success. Being able to hold a conversation, being able to adapt to the work style - which is generally more top-down, directive based than American work culture. These are easier when your at-home culture is closer to the target culture in question. Needless to say, teaching your child a culture that's foreign to you is <em>hard.</em></p> <h3 id="higher-paid-opportunities">Higher paid opportunities</h3> <p>So that leads to an even more vague value proposition, which is the possibility that your maximum earning potential goes up when you learn Chinese. It may be true under very specific circumstances, but I would argue that it's unlikely to affect your wage ceiling. Let's talk about the exceptional cases I can see where this might be true:</p> <p><strong>Entrepreneurship</strong></p> <p>Being able to communicate in Chinese opens up a large potential market for any products or services that you can offer. It also allows you to theoretically tap the Chinese manufacturing and supply chain. So theoretically if you target the Chinese market or if you learn how to manufacture, it would help you a lot.</p> <p>The downside is that it's not necessary. If you're running a business, you have more important problems to deal than learning Chinese. If you already know supply chain management and manufacturing, learning Chinese could be a helpful skill if you achieve a high enough level of fluency, which, frankly, is unlikely given the amount of time you'd have to invest versus the reward.</p> <p><strong>Media</strong></p> <p>You can make it big in the Chinese entertainment business if you know Chinese very well - and you happen to be non-Asian. Non-Asians and mixed Asians are very disproportionately represented in Chinese media as show hosts or guests. The famous comedian 大山 is representative of this kind of trend. Even our random white friends living in China have gotten calls to be on news stations for no discernible reason.</p> <p>Unfortunately, this is largely inaccessible to the majority of my friends, who are Asian. As Asian Americans, we are largely invisible (or <a rel="external" href="https://www.reddit.com/r/MarvelStudiosSpoilers/comments/ngeyts/simu_liu_responds_to_the_chinese_criticism_he_was/">ugly</a> 🤣) to mainland Chinese people. When we are visible, we are seen as defective if we <em>don't</em> speak Chinese. Non-Asians, however, are regularly applauded for speaking even basic Chinese, which creates a positive feedback loop for continuing to learn.</p> <p>In China, non-Asians are exotic and interesting and they represent The West, which China has a complicated relationship with as an ascending super power. Asians are just meh.</p> <h2 id="why-learn-chinese">Why learn Chinese?</h2> <p>Learning Chinese is hard and not necessarily financially rewarding. As parents, we have to question whether the time investment is worth it when the kids are so young, especially with the amount of money that can be poured into it with learning materials, tutors, and immersion schools. And we've seen friends do well learning Chinese starting in high school. So why teach kids, especially young kids, Chinese?</p> <p>Our kids are learning Chinese anyway. Ultimately, it's because the reasons are not financial. And, as you'll see, we think those are the best motivators for the kids, anyway.</p> <p>We struggle with how to teach our kids Chinese because I grew up without a good understanding of my grandparents, and because my wife's parents have a tenuous grasp of English themselves. And Cici's childhood in China gave her an appreciation of the culture that she wants to share with us. It comes down to our cultural connections.</p> <p>Our kids watch a lot of Chinese shows, a few of them being shows that Cici watched as a kid, but also a few new ones. Ironically, cultural connection to China means quite the opposite of being productive in a capitalist culture. The fact that our kids can appreciate Chinese shows means that they can appreciate shows, media, and the arts from an entirely different country. During the pandemic, we have all discovered the importance of turning to the arts to keep our spirits up. They have a lot more opportunities to do so, being able to watch Chinese or English shows.</p> <p>It's just, like, <em>fun</em>, man.</p> <p>Secondly, when I went to college, my aim with learning Chinese was to be able to communicate with my grandparents better. Sadly, Cantonese isn't offered at very many colleges (I think it's just Yale), and my grandparents spoke an even more obscure dialect of Chinese called Toisan (台山话 / Taishan). So due to cultural proximity, Mandarin Chinese was the closest I could get to studying my family's language in college. I took it and thankfully my grandmothers both speak Mandarin to some degree, so it has helped.</p> <p>But it was still over twenty years before I could make myself understood to them using sentences more complex than "I like <noun>." Those twenty years weigh on me. My 奶奶 (paternal grandmother) has passed away, and I'm glad we were able to visit her consistently after the birth of our son, but I want - and cannot have - more time.</p> <p>I don't want that for my kids.</p> <p>While my parents speak Cantonese and Toisan, they speak English by preference now. But Cici's parents are still much more comfortable in Mandarin Chinese. I'm not worried about our kids learning English - they will pick it up. But we are worried that they won't be able to connect with their maternal grandparents linguistically or culturally.</p> <p>And so we make sure to expose them to a good deal of Chinese when we can, even though we speak to each other in English. We put in a consistent amount of moderate effort to do these things.</p> <h2 id="how-we-re-teaching-the-kids-chinese">How we're teaching the kids Chinese</h2> <p>Cici has seen it from the lens of learning English as a foreign language as a child, and I've come at it from the other end, learning Chinese as a foreign language as an adult, and we've converged on the same set of principles. We feel that the key to success is to cultivate the kids' personal reasons for learning Chinese.</p> <p>We both agree that workbooks, passive learning, and other explicit <em>language learning</em> activities are very poor motivators, poor motivation leads to poor engagement, and poor engagement leads to poor retention.</p> <p>Cici remembers her mom turning on the AM radio on long trips in the USA for "passive learning" and my wife would just zone out. When she was bored enough, she would tune in, but wouldn't really catch anything. She has done the workbooks and the exercises and the memorization, but nothing of consequence stuck before moving to the US.</p> <p>Even as recently as just a few years ago, I was writing and rewriting characters to memorize them and I can confidently say I recall almost none of the pages of characters I wrote. In college, I learned thrilling words like "bill of lading" in Chinese, which I would say I forgot but in truth never truly managed to learn. And we both have friends who recounted ruefully that they spent plenty of weekends in Chinese school only to forget it as soon as they stopped, or worse, grew to hate it.</p> <p>But what did work for Cici as a tween was learning lyrics of the songs her friends were into. I still remember Chinese lyrics of songs I learned 15 years ago, and learning songs in English or Chinese is something we still connect on today.</p> <p>The key idea for us was to focus on the content, not the language learning itself. The learning would come if the content was engaging. And so we did our best to craft a miniature digital Chinese immersion environment through media exposure including apps, everyday communication, and a restricted English media diet. Importantly, though their options are limited, they choose what they want to do.</p> <p><strong>Learn Chinese myself</strong></p> <p>First of all, as the parent least comfortable in Chinese, it's my responsibility to try to continue to learn and push myself to learn Chinese. In day to day interactions, I try to use mostly Chinese with our children unless I really just don't know how to say something. And if it bothers me enough, it's a sign that I need to learn the word for it.</p> <p>This is hard with a busy schedule. Theoretically I should keep flash cards for words I should or am currently learning, but that's a little too much overhead for me right now, so I just try to learn with my kids.</p> <p>I think if I had married a partner who spoke a different, non-heritage (for me) language, I would likely try to do the same.</p> <p><strong>Accept that code-switching is a part of life</strong></p> <p>Code switching, or switching between languages, is a perfectly natural part of life, even for native Chinese speakers. Accepting that we'll be mixing Chinese and English makes it easier to work vocabulary into your current patterns of speech. Accepting that your children will code switch makes it easier to accept that some days they'll feel like speaking English, and some days they'll feel like speaking Chinese.</p> <p>Importantly, code switching should not be seen as a deficiency by default. If your kids code switch with each other, they might be reinforcing each others' vocabulary in a mix of languages. And in countries like Malaysia, code switching is part of living in a multilingual country. Cici's Malaysian friend can code switch between Malay, English, and Chinese in the same sentence. It's not because she can't think of the words in each language, she's simply using the words that are the closest proximity to the idea she's trying to express.</p> <p>I will also sometimes code switch on purpose because I know that I can connect the ideas of one language that aren't particularly strong in another. As a contrived example, we use the word 洗澡 or bathe/shower all the time. So if my son points at a car wash and asks "what is that?" I can say it's a 车洗澡的地方 or "a place for cars to bathe." Or when I can easily explain something in English, I start with an explanation or a word in Chinese to let him struggle with the meaning before explaining it in English.</p> <p>That said, I do think patterns of code switching and the inability to think of words in a language are actual weaknesses. We also need to encourage them to speak Chinese, ultimately. It's the speaking of the language that really cements it for the long run. I know this as a child who used to speak Cantonese a lot better than I do now. Once I stopped speaking it, I began to lose the ability to even speak the phonemes correctly, which made me even more self-conscious and less likely to learn.</p> <p><strong>Restrict English media</strong></p> <p>English media is a pervasive fact of life in America, and we try not to have anything to do with it. They will never lack for English media, and the latest and greatest cartoons and animations will be in English, so we do not allow unfettered access to English media.</p> <p>That means no YouTube, time limits on English media apps like WTTW, and supervised access for shows on Disney+, which we watch in the Taiwanese Mandarin localization whenever possible.</p> <p>In America, access to English media will never be a limitation. It can easily become an addiction, though, and for a kid, the language of entertainment quickly becomes the language of preference.</p> <p><strong>Encourage Chinese media</strong></p> <p>That said, there are a lot of Chinese shows <em>on</em> YouTube that we would like for the kids to be able to access, and for that, I use <a rel="external" href="https://github.com/yt-dlp/yt-dlp">yt-dlp</a> to download entire playlists at a time. Then for access, I run a <a rel="external" href="https://jellyfin.org/">Jellyfin</a> instance on my iMac to serve the videos over our router. Yes, being a programmer helps a lot, especially since setting up Jellyfin to be actually usable took a lot of tweaking.</p> <p>Here are some shows on YouTube that we self-host:</p> <ul> <li> <p>宝宝巴士 - Baby Bus: shows, lullabies, and nursery rhymes good for babies and up</p> </li> <li> <p>巧虎 - Qiaohu: lots of Asian-style life lessons. Qiaohu was teaching kids to wear masks 3 years before the Covid-19 pandemic</p> </li> <li> <p>海底小纵队 - Octonauts: a show about saving animals in the ocean that our kids loved</p> </li> <li> <p>小猪佩奇 - Peppa Pig: not the parents' favorite show because the characters can be kind of obnoxious, but the kids like it</p> </li> </ul> <p>Additionally for older kids, our 8yo has enjoyed:</p> <ul> <li> <p>叶罗丽 - Yeluoli, a show about magical dolls and fighting evil</p> </li> <li> <p>小花仙 - Xiaohuaxian, a story about a girl with a magic bracelet</p> </li> <li> <p>绿野仙踪 - The Wizard of Oz: a favorite of Cici's from her childhood. It's an anime remix of the Wizard of Oz that was originally in Japanese, but she watched it as a kid in China</p> </li> <li> <p>还珠格格 - A period drama about a fake princess</p> </li> <li> <p>白蛇 - An animated movie about a demon snake and the snake catcher who finds her - EDIT - I forgot this one’s not a kids’ show. There is definitely a sex scene. That said, it’s pretty subtle and it went over our kids’ heads.</p> </li> </ul> <p>You can filter your YouTube search results for playlists. When you give <code>yt-dlp</code> the URL of a playlist, it will download the entire thing.</p> <p>There are also a few shows / movies that the kids enjoy on Disney+ in the Taiwanese Mandarin localization:</p> <ul> <li> <p>The Lion Guard - a show about one of Simba's sons. Almost the entire show has been localized in Taiwanese Mandarin.</p> </li> <li> <p>Bluey - a show about an Australian dog family. About a third or a fourth of this show has been localized. Update - the entirety of this show, as far as I can tell, has been translated! 🎉</p> </li> </ul> <p>Finally, we used to let the kids watch YouKu as much as they want, but unfortunately there are stupid slapstick humor shows on that as well that our son loves and learns bad habits from. It's still more regulated than YouTube though, so we feel safer letting them have unsupervised access than we do with YouTube.</p> <p>📖 Story time: When our eldest was maybe 3 or 4, we would tell Cici's parents not to let her watch YouTube without them. Time and time again, we would come back to find her watching YouTube on her own. One time - one of the last times - I came downstairs to find her watching YouTube on the television. The video in question involved a giant cartoon baby face that stretched across the length of the TV. The corners of its mouth were held open by raven claws reaching in from the corners of the screen. The baby looked like it was in pain and occasionally shed a tear. At the bottom of the screen, a conveyor belt brought random objects to the baby's mouth which were force-fed to the baby. It was like something out of a feverish nightmare. So, and pardon my French, <em>fuck YouTube's algorithm</em>.</p> <p><strong>Exposure via family</strong></p> <p>We have Cici's mom stay over roughly 4 days out of the week because it would be impossible for us both to work without a third adult to help keep the kids safe, and it helps us expose the kids to Chinese through relatives. We're not always the best at speaking to the kids in Chinese, but Cici's mom is significantly more comfortable speaking Chinese than English, so Chinese remains her default mode of communication. It helps a lot, but she doesn't consistently push them to use Chinese, which is something we're working on. Cici's dad is also a comedic figure and likes Chinese word play.</p> <p>Adorably, when our children see <em>my</em> maternal grandma, they can communicate with her, too. ❤️</p> <p><strong>Apps</strong></p> <p>We have a few different learning apps installed, and they are some of the few that don't have time limits on them, so when they go on long screen time binges, which is unfortunately too often, they end up doing more Chinese apps. One of the most effective is iHuman's 洪恩识字 app. Our son also has a favorite that I'll update this article with when I can find it.</p> <p>EDIT - found them:</p> <ul> <li> <p>iHuman 洪恩识字</p> </li> <li> <p>2Kids 识字</p> </li> <li> <p>2Kids Chinese</p> </li> <li> <p>2Kids 学拼音</p> </li> <li> <p>2Kids 数学天天练</p> </li> </ul> <p>Ideally, try to let the kids use the app in Chinese mode, not English mode for full immersion. That means for Chinese illiterate parents, the UI will be inscrutable. But breaking the immersion is costly. On a meta level, it's not just engagement with the content that's important in this case, but engagement with navigating the content, which helps them navigate content anywhere. Avery has mentioned that she knows characters because she knows what buttons labeled with those characters do. Also a lot of Chinese apps ironically emphasize learning English, which is very annoying.</p> <p><strong>Neglect handwriting</strong></p> <p>We have almost no emphasis on having the kids learn handwriting beyond possibly writing their names. When we teach the kids characters, we let them use pinyin keyboards to write in Chinese. Otherwise this would all take much, much more effort for less reward.</p> <p>That said, the apps they use do have handwriting sections, and if they enjoy it, well, good for them!</p> <h3 id="different-strokes">Different strokes</h3> <p>When our eldest daughter was around 4 years old, she was going to daycare with English-speaking caretakers and gushing over PJ Mask with her friends. I became alarmed when she started responding to her grandparents' Chinese inquiries in English. Part of it no doubt was her interest in hearing their accents in English, but a large part was that she was just becoming more comfortable speaking English than Chinese, <em>even in Chinese language contexts.</em></p> <p>I had lived thirty-plus years of that, and I found the possibility of that outcome intolerable.</p> <p>Now, our oldest daughter's Chinese is <em>decidedly</em> better than mine in every dimension. She's basically literate and able to pick up new characters much more quickly due to her verbal fluency and familiarity with ~1.5k characters.</p> <p>We have been homeschooling our children since the start of the pandemic and are planning on continuing to do so until all of our children can get the Covid-19 vaccine. That means we set the school curriculum, which includes an hour of Chinese using 洪恩识字 each school day, in which Cici emphasizes sentence composition. We incentivize by letting her watch YouKu or Jellyfin afterwards.</p> <p>Her Chinese has really blossomed under these circumstances, and she's eager to read every Chinese character she can find, and she enjoys messaging her 阿婆 (maternal grandma) in Chinese.</p> <p>But I don't think the method of instruction would work with every kid. Our 3 year old son, for instance, would probably hate sitting down to study at any age and I doubt there will ever be an cajoling, incentivizing, or punishment that would get him to do it.</p> <p>Right now, our son is exploring Chinese through the apps that we have and we appreciate that he likes to play on the Chinese learning apps about as much as the English ones.</p> <p>Ultimately, language learning for the kids has to be something that they enjoy.</p> 2022-03-27T12:26:38+00:00 2022-03-27T12:26:38+00:00 Unknown https://briankung.dev/2022/03/27/ruby-has-economy-class-functions/ <p>🇺🇦 <em>I'm joining the Ruby community in calling for an end to Russia's unjust and illegal war in Ukraine! Please donate to Ukrainian efforts <a rel="external" href="https://ua-aid-centers.com/funds-and-foundations">here</a> and learn more <a rel="external" href="https://war.ukraine.ua/">here</a>.</em> 🇺🇦</p> <p>Ruby has economy-class functions, not first-class functions. And that's okay! It's a great language anyway.</p> <p>Let's talk about what first-class functions look like. Per the Wikipedia entry, a programming language is considered to support first-class functions if it:</p> <ol> <li><strong>Non-local variables and closures</strong><br /> Functions that maintains the environment (and therefore the variables) that was in-scope when the function was created.</li> <li><strong>Anonymous and nested functions</strong><br /> Functions that aren't bound to a variable. Why? So we can pass them into other functions without first needing to assign them to a variable. And "nested function" just means a function that is defined while calling another function.</li> <li><strong>Higher order functions</strong><br /> Functions that can take other functions as arguments and return them as well.</li> </ol> <p>The good news is that Ruby supports most of these use cases! But not in a way that I'd consider first-class. Economy, perhaps, but not first-class. Let's talk about each one in turn</p> <h3 id="non-local-variables-and-closures">Non-local variables and closures</h3> <p>Ruby supports this aspect of first-class functions!</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span>a</span><span style="color: #179299;"> =</span><span style="color: #FE640B;"> 1</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">#=&gt; 1</span></span> <span class="giallo-l"><span>b</span><span style="color: #179299;"> =</span><span style="color: #FE640B;"> 2</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">#=&gt; 2</span></span> <span class="giallo-l"><span>d</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> lambda</span><span style="color: #7C7F93;"> {|</span><span>c</span><span style="color: #7C7F93;">|</span><span> a </span><span style="color: #179299;">+</span><span> b </span><span style="color: #179299;">+</span><span> c</span><span style="color: #7C7F93;">}</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">#=&gt; #&lt;Proc:0x0000000108d29d90 (irb):3 (lambda)&gt;</span></span> <span class="giallo-l"><span>d</span><span style="color: #7C7F93;">.(</span><span style="color: #FE640B;">3</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">#=&gt; 6</span></span></code></pre> <p>Verdict: ✅ Arguably Ruby popularized the use of anonymous functions in other languages, even to the extent of <a rel="external" href="https://doc.rust-lang.org/book/ch13-01-closures.html#refactoring-with-closures-to-store-code">inspiring Rust's syntax for anonymous functions.</a></p> <h3 id="anonymous-and-nested-functions">Anonymous and nested functions</h3> <p>Again, another case where Ruby shines. If you've provided a block to an argument, you've used Ruby's fantastic language-level support for anonymous functions:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span>array</span><span style="color: #179299;"> =</span><span style="color: #7C7F93;"> (</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">..</span><span style="color: #FE640B;">15</span><span style="color: #7C7F93;">).</span><span style="color: #1E66F5;font-style: italic;">to_a</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">#=&gt; [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]</span></span> <span class="giallo-l"><span> 👇 anonymous function!</span></span> <span class="giallo-l"><span>array</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">select</span><span style="color: #7C7F93;"> {|</span><span>number</span><span style="color: #7C7F93;">|</span><span> number</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">even?</span><span style="color: #7C7F93;">}</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">#=&gt; [2, 4, 6, 8, 10, 12, 14]</span></span></code></pre> <p>Nesting functions is a less common pattern, but it <em>is</em> slightly more awkward, and you have to <em>know things</em>:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span style="color: #8839EF;">def</span><span style="color: #1E66F5;font-style: italic;"> uses_a_function</span><span style="color: #7C7F93;">(</span><span style="color: #E64553;font-style: italic;">func</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span> 👇 but why?</span></span> <span class="giallo-l"><span> func</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">call</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">2</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #8839EF;">end</span></span> <span class="giallo-l"><span> 👇 🤔</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;">uses_a_function</span><span style="color: #7C7F93;">(</span><span style="color: #1E66F5;font-style: italic;">-&gt;</span><span>(</span><span style="color: #E64553;font-style: italic;">a</span><span style="color: #7C7F93;">,</span><span style="color: #E64553;font-style: italic;"> b</span><span>)</span><span style="color: #7C7F93;"> {</span><span>a </span><span style="color: #179299;">+</span><span> b</span><span style="color: #7C7F93;">})</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">#=&gt; 3</span></span></code></pre> <p>If this particular <em>knowing of things</em> is troubling to you, it should give you a sense of deep foreboding for the next section.</p> <p>Verdict: ✅ who needs nested functions, anyway?</p> <h3 id="higher-order-functions">Higher order functions</h3> <p>Next, let's try to return functions from functions in Ruby:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span style="color: #8839EF;">def</span><span style="color: #1E66F5;font-style: italic;"> returns_function</span></span> <span class="giallo-l"><span style="color: #8839EF;"> def</span><span style="color: #1E66F5;font-style: italic;"> addition</span><span style="color: #7C7F93;">(</span><span style="color: #E64553;font-style: italic;">a</span><span style="color: #7C7F93;">,</span><span style="color: #E64553;font-style: italic;"> b</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span> a </span><span style="color: #179299;">+</span><span> b</span></span> <span class="giallo-l"><span style="color: #8839EF;"> end</span></span> <span class="giallo-l"><span style="color: #8839EF;">end</span></span></code></pre> <p>Easy! Well, not really:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span>add</span><span style="color: #179299;"> =</span><span> returns_function</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">#=&gt; :addition</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;">add</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">2</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;">(</span><span>irb</span><span style="color: #7C7F93;">):</span><span style="color: #FE640B;">10</span><span style="color: #7C7F93;">:</span><span style="color: #D20F39;">in</span><span style="color: #40A02B;"> `&lt;main&gt;&#39;: undefined method `</span><span>add</span><span style="color: #40A02B;">&#39; for main:Object (NoMethodError)</span></span></code></pre> <p>You can do this pretty handily in Javascript:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="javascript"><span class="giallo-l"><span style="color: #8839EF;">let</span><span style="color: #1E66F5;font-style: italic;"> returnsFunction</span><span style="color: #179299;"> =</span><span style="color: #8839EF;"> function</span><span style="color: #7C7F93;">() {</span></span> <span class="giallo-l"><span style="color: #8839EF;"> return function</span><span style="color: #7C7F93;">(</span><span style="color: #E64553;font-style: italic;">a</span><span style="color: #7C7F93;">,</span><span style="color: #E64553;font-style: italic;">b</span><span style="color: #7C7F93;">) {</span><span style="color: #8839EF;">return</span><span> a</span><span style="color: #179299;"> +</span><span> b</span><span style="color: #7C7F93;"> };</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span> <span class="giallo-l"><span style="color: #8839EF;">let</span><span> addition</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> returnsFunction</span><span>()</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;">addition</span><span>(</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">2</span><span>)</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">// 3</span></span></code></pre> <p>You <em>can</em> do this in Ruby, it's just more...<em>economy</em> class. The aisles are tighter and you have a longer line to board:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span style="color: #8839EF;">def</span><span style="color: #1E66F5;font-style: italic;"> returns_function</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Proc</span><span style="color: #7C7F93;">.</span><span style="color: #8839EF;">new do</span><span style="color: #7C7F93;"> |</span><span>a</span><span style="color: #7C7F93;">,</span><span>b</span><span style="color: #7C7F93;">|</span></span> <span class="giallo-l"><span> a </span><span style="color: #179299;">+</span><span> b</span></span> <span class="giallo-l"><span style="color: #8839EF;"> end</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"> # or you could use a lambda:</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;"> -&gt;</span><span>(</span><span style="color: #E64553;font-style: italic;">a</span><span style="color: #7C7F93;">,</span><span style="color: #E64553;font-style: italic;">b</span><span>)</span><span style="color: #7C7F93;"> {</span><span>a </span><span style="color: #179299;">+</span><span> b</span><span style="color: #7C7F93;">}</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"> # ...which is shorthand for:</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;"> lambda</span><span style="color: #7C7F93;"> {|</span><span>a</span><span style="color: #7C7F93;">,</span><span>b</span><span style="color: #7C7F93;">|</span><span> a </span><span style="color: #179299;">+</span><span> b</span><span style="color: #7C7F93;">}</span><span> </span></span> <span class="giallo-l"><span style="color: #8839EF;">end</span></span> <span class="giallo-l"><span>addition</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> returns_function</span><span style="color: #7C7F93;">()</span></span></code></pre> <p>And then you can use it like a normal function...except that you can't because it's a <code>Proc</code> (or a <code>Lambda</code>) and not a method as supported by the language:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span style="color: #1E66F5;font-style: italic;">irb</span><span style="color: #7C7F93;">(</span><span>main</span><span style="color: #7C7F93;">):</span><span style="color: #FE640B;">008</span><span style="color: #7C7F93;">:</span><span style="color: #FE640B;">0</span><span style="color: #179299;">&gt;</span><span style="color: #1E66F5;font-style: italic;"> addition</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">2</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;">(</span><span>irb</span><span style="color: #7C7F93;">):</span><span style="color: #FE640B;">8</span><span style="color: #7C7F93;">:</span><span style="color: #D20F39;">in</span><span style="color: #40A02B;"> `&lt;main&gt;&#39;: undefined method `</span><span>addition</span><span style="color: #40A02B;">&#39; for main:Object (NoMethodError)</span></span></code></pre> <p>So you have to get back in the economy class line:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span>addition</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">call</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">2</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">#=&gt; 3</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"># or the shorthand version:</span></span> <span class="giallo-l"><span>addition</span><span style="color: #7C7F93;">.(</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">2</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">#=&gt; 3</span></span></code></pre> <p>Alternately, you can use the <code>def...end</code> syntax in Ruby, as long as you know that a method definition returns the method's <code>Symbol</code> instead of the actual <code>Method</code> object:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="ruby"><span class="giallo-l"><span style="color: #8839EF;">def</span><span style="color: #1E66F5;font-style: italic;"> returns_function</span></span> <span class="giallo-l"><span style="color: #8839EF;"> def</span><span style="color: #1E66F5;font-style: italic;"> addition</span><span style="color: #7C7F93;">(</span><span style="color: #E64553;font-style: italic;">a</span><span style="color: #7C7F93;">,</span><span style="color: #E64553;font-style: italic;"> b</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span> a </span><span style="color: #179299;">+</span><span> b</span></span> <span class="giallo-l"><span style="color: #8839EF;"> end</span></span> <span class="giallo-l"><span style="color: #8839EF;">end</span></span> <span class="giallo-l"><span>add</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> returns_function</span><span style="color: #7C7F93;">()</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">#=&gt; :addition</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"># Note that calling returns_function has actually defined the addition method in the caller&#39;s scope, but let&#39;s pretend we didn&#39;t know that</span></span> <span class="giallo-l"><span>add</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">class</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">#=&gt; Symbol</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"># Now we go from a Symbol to a method with the Object#method class:</span></span> <span class="giallo-l"><span>add</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> method</span><span style="color: #7C7F93;">(</span><span>add</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #179299;">=&gt;</span><span style="color: #7C7F93;font-style: italic;"> #&lt;Method: Object#addition(a, b) (irb):2&gt;</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"># And now we can finally call it:</span></span> <span class="giallo-l"><span>add</span><span style="color: #7C7F93;">.(</span><span style="color: #FE640B;">1</span><span style="color: #7C7F93;">,</span><span style="color: #FE640B;">2</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">#=&gt; 3</span></span></code></pre> <p>You <em>can</em> do it, but the obstacles with the language dissuade you from currying, passing functions around, and calling them. Furthermore, even if you were to shoehorn Ruby into a first-class function usage patterns, you would have to know the differences between methods, procs, and lambdas, and how to call them. As a result, the large majority of the Ruby ecosystem does not use functions in a first-class manner, which is disappointing because the support for it is <em>so close.</em></p> <p>A large portion of support would be getting <em>method references</em>, i.e. a shorthand for <code>Object#method</code> which would let us refer to methods without the ceremony of calling <code>method(:some_method's_symbol)</code>. In fact, support for method references was added at some point, but then removed. So instead of calling <code>method(:some_method's_symbol)</code>, the proposal was to create a language level syntactical sugar for it using <code>.:some_method's_symbol</code>. And here is the feature request in Ruby's bug tracker: <a rel="external" href="https://bugs.ruby-lang.org/issues/13581#note-45">https://bugs.ruby-lang.org/issues/13581#note-45</a>.</p> <p>The other part would be the awkwardness around calling the returned method, but that seems unavoidable at this point due to the ambiguity caused by Ruby not requiring parentheses in order to call a method.</p> <p>Verdict: ⚠️ technically yes, but realistically no. It's not great and it doesn't lead to the sorts of usage patterns that you see in other languages with first-class functions.</p> <p><em>Note: I have to point out that the <code>.:</code> syntax was proposed by <a rel="external" href="https://twitter.com/zverok">Victor Shepelev aka zverok</a>, who is a Ukrainian Ruby core member who is currently sheltering in Kharkiv, one of the many cities under attack by Russian forces. Read recent developments <a rel="external" href="https://zverok.space/blog/2022-03-15-STILL-WAR.html">here</a>, and zverok's blog <a rel="external" href="https://zverok.space/writing/#blog">here</a>.</em></p> 2022-02-02T18:02:14+00:00 2022-02-02T18:02:14+00:00 Unknown https://briankung.dev/2022/02/02/writing-a-rust-gem-from-scratch/ <p><em>Note: anytime you see</em> 🙋‍♂️ RFC_, that's a "Request For Comments" about a topic I didn't understand or take the time to look into. Please feel free to add what you know!_</p> <p>This is a followup to the <a rel="external" href="https://briankung.dev/2022/01/31/sneak-preview-writing-ruby-gem-native-extensions-in-rust/">last post.</a> Instead of using the template <code>rust_ruby_example</code> gem, we'll make one from scratch. Make sure to go back over the <a rel="external" href="https://briankung.dev/2022/01/31/sneak-preview-writing-ruby-gem-native-extensions-in-rust/#using-a-rubygems-fork">"Using a <code>rubygems</code> fork"</a> section because we'll be using it heavily during this post, as well!</p> <h2 id="requirements">Requirements</h2> <p>Requirements / dependencies / utilities I used and their versions on macOS Monterey v12.1 (21C52) as of 2022-02-02:</p> <ul> <li>Bundler version 2.4.0.dev</li> <li>gem version 3.4.0.dev</li> <li>cargo 1.58.0 (f01b232bc 2022-01-19)</li> <li>rustc 1.58.1 (db9d1b20b 2022-01-20)</li> </ul> <h2 id="generating-a-new-gem">Generating a new gem</h2> <p>Let's see what it takes to write a Rust gem from scratch. Thankfully, Bundler has a <a rel="external" href="https://bundler.io/guides/creating_gem.html">generator for making new gems</a>, and we can look at the <code>[rust_ruby_example](https://github.com/briankung/rust_ruby_example)</code> gem for pointers on how to get the Rust parts working.</p> <p>We're still going to be doing some string manipulation, but this time we'll just shuffle the characters. Full disclosure: I also don't know enough of Ruby's C API to do much more than that. That's an adventure for another day!</p> <p>Let's start out by asking <code>bundler</code> to make a new gem. We'll call it <strong>r</strong>ust_sh<strong>uffle</strong>, or <strong><code>ruffle</code></strong> for short.</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-000-sh">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-000-sh</a></p> <p>If this is the first time you've created a new gem with <code>bundler</code>, it may ask you a few configuration questions first. These questions are saved to your user's profile at <code>~/.bundle/config</code> and can be changed with the <code>bundle config</code> subcommand. For our purposes, the only one that matters is using <code>rspec</code> as the test framework.</p> <p>Speaking of <code>rspec</code>, let's <code>$ bundle install</code> in our new <code>ruffle</code> directory so we can fetch the <code>rspec</code> gem:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-001-sh">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-001-sh</a></p> <p>Well! Looks like we'll be using some good old-fashioned EDD (Error Driven Development) to get this gem ship-shape.</p> <p>The problem here is that the gemspec, a metadata file, needs to be filled out before the gem is valid. We're going to do it the easy way and simply remove all the <code>TODO</code>'s from the. Bundler also checks that the URLs parse correctly, so we'll be replacing all the URLs with <code>"https://example.com"</code>:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-002-rb">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-002-rb</a></p> <p>And now we can fetch our dependencies:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-003-sh">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-003-sh</a></p> <p>Now, what happens when we run our specs?</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-004-sh">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-004-sh</a></p> <p>The test "Ruffle does something useful" in the file <code>spec/ruffle_spec.rb</code> on line 8 fails. Harsh.</p> <h2 id="adding-a-shuffle-method-in-rust">Adding a <code>#shuffle</code> method in Rust</h2> <p>First, let's change the "does something useful" test to <em>test</em> something useful. Replace <code>spec/ruffle_spec.rb</code> with the following:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-005-rb">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-005-rb</a></p> <p>Now let's run the test, like good EDD practitioners - red, green, deploy. Then right back to EDD, this time in prod (it's a virtuous cycle):</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-006-sh">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-006-sh</a></p> <p>You may see a warning like <code>--pattern spec/**{,/*/**}/*_spec.rb failed</code>. This is a result of <a rel="external" href="https://stackoverflow.com/questions/36427818/rspec-showing-error-message">running <code>rspec</code> through <code>rake</code>.</a> While it's annoying, it's not a showstopper.</p> <h3 id="initializing-a-rust-project">Initializing a Rust project</h3> <p>Now let's add some Rust code! For the purposes of this tutorial, we'll add it directly to the root directory of the gem, but there is a <a rel="external" href="https://github.com/rake-compiler/rake-compiler#using-a-standardized-project-structure">standard project structure</a> for gem native extensions.</p> <p>(🙋‍♂️ RFC: should a Rust extension follow the <a rel="external" href="https://github.com/rake-compiler/rake-compiler#using-a-standardized-project-structure"><code>rake-compiler</code> project structure</a>?)</p> <p>You can initialize a Rust project with:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-007-sh">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-007-sh</a></p> <p>We pass the <code>--lib</code> argument to <code>cargo</code> to tell it that we want our crate to be a library, not a binary. Whereas a binary results in an executable, a library crate's output should be something like <code>.so</code> or <code>.dll</code> files, if not compiled directly into another Rust binary.</p> <p>Now we have two new files: <code>Cargo.toml</code> and <code>src/lib.rs</code>. <code>cargo</code> has also modified the <code>.gitignore</code> file to ignore build artifacts and <code>Cargo.lock</code>, a lockfile for Cargo dependencies.</p> <h3 id="adding-a-rake-compile-task">Adding a <code>rake compile</code> task</h3> <p>Let's add a <code>rake</code> task to compile our Rust code. <code>Rakefile</code>s are Ruby files that define the tasks that <code>rake</code> can run. If we inspect ours, it looks fairly barebones:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-008-rb">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-008-rb</a></p> <p>Basically it requires some default tasks. It also defines the <code>:spec</code> task as the default task, or the task that's run when you execute <code>rake</code> without any arguments. We've already used it, in fact.</p> <p>Let's add a <code>compile</code> task. Make sure the <code>RUBYGEMS_PATH</code> environment variable is set <a rel="external" href="https://briankung.dev/2022/01/31/sneak-preview-writing-ruby-gem-native-extensions-in-rust/#aliasing-your-default-rubygems">from the last post</a>. If you don't have it, make sure to export it:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-013-sh">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-013-sh</a></p> <p>Now we can reference that:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-014-3-rb">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-014-3-rb</a></p> <p>💡 <strong>Click</strong> for additional notes - <code>*cargo_builder_gem</code>?</p> <hr /> <p>We have to make sure we're using the <code>cargo-builder</code> branch of rubygems. If we simply shell out to <code>gem</code>, we'll end up using our default system gem.</p> <p>I was hoping to use the <code>rubygem</code> internals to build the gem instead of relying on shelling out to the utility. I got as far as to <code>require 'rubygems/ext'</code> in order to use the <code>Gem::Ext::CargoBuilder</code> class, but realized that <code>gem</code> and <code>bundler</code> are just aliases to the <code>cargo-builder</code> branch of our <code>rubygems</code> repo and not a system-wide installation, I don't have access to it within Ruby itself. It will be much easier once the PR is merged into the repository proper.</p> <p>Now, there is a way using the <code>setup.rb</code> script in the <code>rubygems</code> repository, but it requires replacing your default <code>rubygems</code> gems. As long as this article is, it would be even longer with the caveats and restoration that would take, so I chose not to use <code>setup.rb</code>.</p> <hr /> <p>And now to test:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-015-sh">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-015-sh</a></p> <p>💡 <strong>Click</strong> for additional notes - <code>"true"</code>?</p> <hr /> <p><strong>EDIT</strong> - Thanks Zach! I've changed it to backticks 😊</p> <p>The call to check whether the <code>ruffle</code> gem is installed is <code>system 'gem', 'list', '-i', '^ruffle$'</code>. Unfortunately, the actual shell command prints <code>true</code> or <code>false</code> to stdout, so I can't swallow it by, for example, assigning it to a variable. I'm too lazy and this post has taken too long already to add this to the list of things to figure out.</p> <p>My apologies 🙇‍♂️</p> <p>🙋‍♂️ RFC: what's a better way to do this?</p> <hr /> <p>At this point, however, it's not actually building the extension. If it were, it would print the message <code>"Building native extensions. This could take a while..."</code>.</p> <h3 id="fixing-crate-compilation">Fixing crate compilation</h3> <p>There are two more things we need to change in <code>ruffle.gemspec</code> before it'll work. First, we have to add <code>"Cargo.toml"</code> to <code>spec.extensions</code>:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-016-rb">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-016-rb</a></p> <p>But this still isn't enough. If we run <code>$ rake compile</code> now, we get a cryptic error message:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-017-sh">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-017-sh</a></p> <p>The problem is that the we aren't telling the gemspec to package the <code>Cargo.lock</code> file in the final <code>.gem</code> file, and building the crate requires a lock on the <code>Cargo.lock</code> file. So how do we add the lockfile to the finished gem? By adding them to the <code>spec.files</code> array in the gemspec. You can read more about that <a rel="external" href="https://guides.rubygems.org/specification-reference/#files">here.</a></p> <p>There are two ways we can do this:</p> <ol> <li>Add the <code>Cargo.lock</code> and <code>Cargo.toml</code> files to the <code>git</code> history. The default logic for <code>spec.files</code> relies on the <code>git</code> history. It uses <code>git ls-files -z.split("\x0")</code>, which only reports files that have been added to <code>git</code>.</li> <li>Specifically add <code>["Cargo.lock", "Cargo.toml", and "src/lib.rs"]</code> to <code>spec.files</code> in the gemspec.</li> </ol> <p>Since we haven't been paying much attention to <code>git</code> and this is a simple enough project with no Ruby files, we'll go with option #2. In <code>ruffle.gemspec</code>, find the code that assigns to <code>spec.files</code>:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-018-rb">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-018-rb</a></p> <p>...and replace it with this:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-019-rb">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-019-rb</a></p> <p>We need to set the <code>crate-type</code> to <code>"cdylib"</code> in order to tell the Rust compiler that the output should be a shared library that can be used from other languages. As per the Rust docs on <a rel="external" href="https://doc.rust-lang.org/reference/linkage.html">Linkage</a>:</p> <p>&gt; <code>--crate-type=cdylib</code>, <code>#[crate_type = "cdylib"]</code> - A dynamic system library will be produced. This is used when compiling a dynamic library to be loaded from another language. This output type will create <code>*.so</code> files on Linux, <code>*.dylib</code> files on macOS, and <code>*.dll</code> files on Windows.</p> <p>So next, we add <code>crate-type</code> to <code>Cargo.toml</code>:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-021-toml">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-021-toml</a></p> <p>Now we can finally count on <code>rake</code> to build the crate:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-022-sh">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-022-sh</a></p> <p>💡 <strong>Click</strong> for additional details - what happens when we don't set <code>crate-type</code>?</p> <hr /> <p>When we run <code>$ rake compile</code>, we get another error:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>$ rake compile</span></span> <span class="giallo-l"><span># ...omitted</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>Compiling ruffle v0.1.0 (/Users/brian/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/ruffle-0.1.0)</span></span> <span class="giallo-l"><span> Finished release [optimized] target(s) in 0.47s</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>Dynamic library not found for Rust extension (in /Users/brian/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/extensions/x86_64-darwin-21/3.0.0/ruffle-0.1.0)</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>Make sure you set &quot;crate-type&quot; in Cargo.toml to &quot;cdylib&quot;</span></span></code></pre> <p>The error message came with instructions this time! <a rel="external" href="https://github.com/ianks/rubygems/blob/cargo-builder/lib/rubygems/ext/cargo_builder.rb#L287-L301">Thanks, @ianks!</a></p> <hr /> <h3 id="defining-the-ruffle-module-in-rust">Defining the <code>Ruffle</code> module in Rust</h3> <p>While the crate is compiling now, our tests are still failing because we haven't actually done anything with the Rust code. Since we'll be creating the Ruby data structures on the Rust side, we'll use a Ruby API to interface with Ruby internals. Enter <code>[rb-sys](https://github.com/ianks/rb-sys)</code>, Rust bindings for ruby that have been automatically generated using <code>[rust-bindgen](https://github.com/rust-lang/rust-bindgen)</code>.</p> <p>Add it under your dependencies in <code>Cargo.toml</code>:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-023-toml">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-023-toml</a></p> <p>Next, we'll just copy from <code>[rust_ruby_example/src/lib.rs](https://github.com/briankung/rust_ruby_example/blob/main/src/lib.rs)</code>, the example gem from the last post, except we'll substitute "ruffle" wherever it says "rust_ruby_example" and "shuffle" wherever it says "reverse."</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-024-rs">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-024-rs</a></p> <p>Now we just remove the <code>lib/ruffle.rb</code> file:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-026-sh">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-026-sh</a></p> <p>💡 <strong>Click</strong> for additional details - why delete <code>lib/ruffle.rb</code>?</p> <hr /> <p>The <code>respond_to</code> test won't pass because Ruby's <code>require</code> method will look in <code>lib</code> first and find the empty <code>lib/ruffle.rb</code> file before it searches available gems (you can read more about how Ruby's <code>require</code> works here: <a rel="external" href="https://ryanbigg.com/2017/11/how-require-loads-a-gem">https://ryanbigg.com/2017/11/how-require-loads-a-gem</a>).</p> <p>Here's the proof:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>$ rake compile spec</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span># ...omitted</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>Ruffle</span></span> <span class="giallo-l"><span> has a #shuffle method (FAILED - 1)</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>Failures:</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span> 1) Ruffle has a #shuffle method</span></span> <span class="giallo-l"><span> Failure/Error: expect(Ruffle).to respond_to(:shuffle)</span></span> <span class="giallo-l"><span> expected Ruffle to respond to :shuffle</span></span> <span class="giallo-l"><span> # ./spec/ruffle_spec.rb:5:in `block (2 levels) in &lt;top (required)&gt;&#39;</span></span></code></pre> <hr /> <p>(Note that this leaves <code>lib/ruffle/version.rb</code>, which could be confusing for anyone reading your code.)</p> <p>Now, let's run <code>rake spec</code>:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-027-sh">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-027-sh</a></p> <p>We're green! ✅ And all with Rust code.</p> <h3 id="implementing-ruffle-shuffle">Implementing <code>Ruffle#shuffle</code></h3> <p>Next, let's add a test that describes the behavior of the method:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-028-rb">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-028-rb</a></p> <p>Technically, this could fail if the shuffled string randomly returns the original string. If this happens, you've won the random number generator lottery! Take a screenshot 📸</p> <p>And when we run <code>rake spec</code>:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-029-sh">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-029-sh</a></p> <p>Because we've simply copied over <code>rust_ruby_example</code>'s <code>pub_reverse</code> code, it's just reversing the string.</p> <p>Let's see if we can modify our <code>pub_shuffle</code> method in Rust to look like what we want. Right now we have:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-030-rs">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-030-rs</a></p> <p>Instead of <code>let shuffled = ruby_string.chars().rev().collect::&lt;String&gt;()</code> on line 4, we would want something like <code>let shuffled = ruby_string.chars().shuffle().collect::&lt;String&gt;()</code>. Unfortunately, there is no such method implemented on Rust's <code>Iter</code> struct.</p> <p>As per <a rel="external" href="https://stackoverflow.com/a/26035435/1042144">this StackOverflow answer</a>, you can use Rust's <code>rand::seq::SliceRandom</code> trait to provide a <code>shuffle</code> method on <code>Vec</code>s. StackOverflow user Vladimir Matveev's answer looks like this:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-031-rs">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-031-rs</a></p> <p>Shuffling a vector is a randomized operation, so we need a random number generator (RNG), and Rust doesn't have an RNG in its standard library. The de facto standard RNG in Rust is <code>[rand](https://crates.io/crates/rand)</code>. Let's add it to our dependencies:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-032-toml">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-032-toml</a></p> <p>In the original code, the <a rel="external" href="https://doc.rust-lang.org/std/primitive.char.html"><code>char</code>s</a> are reversed and collected into an owned <code>String</code>.</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-033-rs">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-033-rs</a></p> <p>However, we need to actually mutate the <code>Vec</code> in place instead of using fancy functional <code>Iter</code> methods.</p> <p>In <code>rust_ruby_example</code> the <code>input</code> goes from a Ruby <code>VALUE</code> type:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-034-rs">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-034-rs</a></p> <p>To a <code>CString</code> type with this function call:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-035-rs">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-035-rs</a></p> <p>Finally to a Rust <code>String</code> type with the final function call:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-036-rs">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-036-rs</a></p> <p>We want to shuffle the characters of the string, so we'll declare it as such and request the characters. Calling <code>collect()</code> charges you casts the value into the requested type signature if an <code>Into</code> trait is available for the type conversion:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-037-rs">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-037-rs</a></p> <p>💡 Additional notes about <code>chars()</code></p> <hr /> <p>Note that Rust's <code>chars()</code> method does not handle Unicode <em>grapheme clusters</em> (<a rel="external" href="https://briankung.dev/2022/02/02/writing-a-rust-gem-from-scratch/#comment-273">thanks, Wesley!</a>). As per Rust's documentation on the <code>chars</code> method:</p> <blockquote> <p>Remember, <a rel="external" href="https://doc.rust-lang.org/stable/std/primitive.char.html"><code>char</code></a>s might not match your intuition about characters:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"></span> <span class="giallo-l"><span style="color: #8839EF;"> let</span><span> y</span><span style="color: #179299;"> =</span><span style="color: #40A02B;"> &quot;y̆&quot;</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #8839EF;"> let mut</span><span> chars</span><span style="color: #179299;"> =</span><span> y</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">chars</span><span style="color: #7C7F93;">();</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;"> assert_eq!</span><span style="color: #7C7F93;">(</span><span style="color: #DF8E1D;font-style: italic;">Some</span><span style="color: #7C7F93;">(&#39;</span><span style="color: #40A02B;">y</span><span style="color: #7C7F93;">&#39;),</span><span> chars</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">next</span><span style="color: #7C7F93;">());</span><span style="color: #7C7F93;font-style: italic;"> // not &#39;y̆&#39;</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;"> assert_eq!</span><span style="color: #7C7F93;">(</span><span style="color: #DF8E1D;font-style: italic;">Some</span><span style="color: #7C7F93;">(&#39;</span><span style="color: #EA76CB;">\u{0306}</span><span style="color: #7C7F93;">&#39;),</span><span> chars</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">next</span><span style="color: #7C7F93;">());</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;"> assert_eq!</span><span style="color: #7C7F93;">(</span><span style="color: #DF8E1D;font-style: italic;">None</span><span style="color: #7C7F93;">,</span><span> chars</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">next</span><span style="color: #7C7F93;">());</span></span> <span class="giallo-l"><span> </span></span></code></pre></blockquote> <p>The more or less canonical way to handle this is to use the <code>unicode-segmentation</code> crate, as per <a rel="external" href="https://stackoverflow.com/a/58770681/1042144">this StackOverflow answer.</a></p> <p>This is also a handy introduction to Unicode: <a rel="external" href="https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/">The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)</a></p> <hr /> <p>Now we can bring the <code>rand</code> functionality into scope at the top of our file:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-038-rs">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-038-rs</a></p> <p>And then call <code>shuffle</code> on the <code>chars</code> vec:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-039-rs">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-039-rs</a></p> <p>So your function should now look like this:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-040-rs">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-040-rs</a></p> <p>Now we need to convert the <code>Vec&lt;char&gt;</code> to a Rust <code>String</code> and store its length as a <code>c_long</code> type, which in this case is just a type alias for <code>i64</code>:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-041-rs">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-041-rs</a></p> <p>Then we construct a new <code>CString</code> from the shuffled Rust <code>String</code>:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-042-rs">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-042-rs</a></p> <p>And finally return the Ruby String:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-043-rs">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-043-rs</a></p> <p>...and with that, the final function looks like this:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-044-rs">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-044-rs</a></p> <p>And don't forget to check your <code>Init_ruffle</code> method that actually initializes the Ruby <code>Ruffle</code> module and defines the method:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-045-rs">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-045-rs</a></p> <p>(It's unmodified from our <a href="https://briankung.dev/2022/02/02/writing-a-rust-gem-from-scratch/#copy-pasta">earlier template code</a>.)</p> <p>Now to recompile it and run the specs:</p> <p><a rel="external" href="https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-046-sh">https://gist.github.com/briankung/11e2a18b099c4d8b60ac8b519f7cef52#file-046-sh</a></p> <p>And we're passing!</p> <hr /> <p>I'm sure there will be a lot of patterns emerging on how to organize the code once people start creating their own Rust gems - this is by no means a definitive one, just the one I copied from @ianks's <code>rust_ruby_example</code> gem. I'm a novice as well! But hopefully you got as much out of reading this as I did out of writing it. And with <a rel="external" href="https://github.com/rubygems/rubygems/pull/5175#issuecomment-1027085401">@ianks's pull request approved and passing CI</a>, hopefully it'll be only a matter of time before everyone gets to play with the new functionality!</p> <p>The commit messages are pretty bad because it was a spike for me, but you can find all the code here: <a rel="external" href="https://github.com/briankung/ruffle">https://github.com/briankung/ruffle</a></p> 2022-01-31T06:46:52+00:00 2022-01-31T06:46:52+00:00 Unknown https://briankung.dev/2022/01/31/sneak-preview-writing-ruby-gem-native-extensions-in-rust/ <p><em>Note: anytime you see</em> 🙋‍♂️ RFC_, that's a "Request For Comments" about a topic I didn't understand or take the time to look into. Please feel free to add what you know!_</p> <p><em>If this post tickles your fancy, check out the follow-up post: <a rel="external" href="https://briankung.dev/2022/02/02/writing-a-rust-gem-from-scratch/">Writing a Rust gem from scratch</a></em></p> <p>In December 2021, Ian Ker-Seymer (<a rel="external" href="https://github.com/ianks">@ianks</a>) submitted <a rel="external" href="https://github.com/rubygems/rubygems/pull/5175">a pull request</a> to enable native extensions in Rust!</p> <p>I was so excited, I had to try it out, even though it hadn't been merged yet. A lot of maintainers are showing interest and pitching in, so I have high hopes for it being merged into main. So here are my notes on writing a Rust gem extension.</p> <h2 id="requirements">Requirements</h2> <p>Requirements / dependencies / utilities I used and their versions on macOS Monterey v12.1 (21C52) as of 2022-01-29:</p> <ul> <li>Bundler version 2.4.0.dev</li> <li>gem version 3.4.0.dev</li> <li>cargo 1.58.0 (f01b232bc 2022-01-19)</li> <li>rustc 1.58.1 (db9d1b20b 2022-01-20)</li> </ul> <h2 id="using-a-rubygems-fork">Using a <code>rubygems</code> fork</h2> <p><strong>Warning! The following is based on ianks's development branch of Rubygems. The feature may have changed - or not exist at all - by the time you read this. I'll modify this warning if the feature ends up being merged.</strong></p> <p>Find somewhere cozy to clone @ianks's <code>cargo-builder</code> branch of <code>rubygems</code> and run the following:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-000-sh">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-000-sh</a></p> <h3 id="aliasing-your-default-rubygems">Aliasing your default rubygems</h3> <p>We need to be able to use the <code>gem</code> and <code>bundle</code> commands from the <code>cargo-builder</code> branch of <code>rubygems</code>. As per the directions in the <a rel="external" href="https://github.com/rubygems/rubygems/blob/master/CONTRIBUTING.md#getting-started"><code>CONTRIBUTING.md</code></a>:</p> <blockquote> <p>To run commands like <code>gem install</code> from the repo:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>ruby -Ilib bin/gem install</span></span></code></pre> <p>To run commands like <code>bundle install</code> from the repo:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>ruby bundler/spec/support/bundle.rb install</span></span></code></pre></blockquote> <p>But this is a hassle, so we'll use aliases instead of typing all of this up every time. <code>cd</code> into your <code>rubygems</code> directory and alias these commands with the following:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-001-sh">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-001-sh</a></p> <p>We're going to use the <code>RUBYGEMS_PATH</code> variable later on, so keep that handy! Now if you check the version numbers of your default gems, they should be as follows:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-002-sh">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-002-sh</a></p> <p>Note that these aliases won't be present in a new terminal shell!</p> <h2 id="compiling-an-example-gem">Compiling an example gem</h2> <p>We're ready to test the functionality of a Rust-based gem. For starters, let's use the <a rel="external" href="https://github.com/briankung/rust_ruby_example"><code>rust_ruby_example</code> gem</a> that I've extracted from @ianks's pull request:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-003-sh">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-003-sh</a></p> <p>Let's confirm that it does, indeed, allow us to run Rust code from Ruby.</p> <p>First, we need to build the gem. We do this by pointing the <code>gem</code> command at a <code>.gemspec</code> file. Luckily, the repo has one of those:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-004-sh">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-004-sh</a></p> <p>We also explicitly name the output file, otherwise we get something like <code>rust_ruby_example-0.1.0.gem</code>, which is just a tad bit more awkward.</p> <p>And we're done!</p> <p>...well, not exactly. As it turns out, extensions aren't compiled until you <em>install</em> the gem. It makes sense that building the gem and installing the gem are two separate steps. So next we need to install it:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-005-sh">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-005-sh</a></p> <p>Firing up <code>cargo</code> took a minute or so on my machine.</p> <h4 id="potential-errors">Potential errors</h4> <p>Speaking of <code>cargo</code>, if you don't have it installed, you may see a message that looks like this:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-006-sh">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-006-sh</a></p> <p>This means that you don't have cargo installed, or <code>rubygems</code> couldn't find <code>cargo</code> in your <code>$PATH</code>. Make sure to <a rel="external" href="https://www.rust-lang.org/tools/install">install Rust</a> and come back when you're done!</p> <p>You may also see an error like this:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-007-sh">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-007-sh</a></p> <p>The key here being the message <code>"No builder for extension 'Cargo.toml'."</code> If that's the case, double check your <code>bundler --version</code> and <code>gem --version</code> to make sure they match the versions above. Your current version of the <code>gem</code> utility is missing @ianks's <code>CargoBuilder</code> addition.</p> <h3 id="inspecting-rust-ruby-example-code">Inspecting <code>rust_ruby_example</code> code</h3> <p><code>rust_ruby_example</code> includes some sample code in <code>src/lib.rs</code>:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-008-rs">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-008-rs</a></p> <p>If you've never seen Ruby internal code before, a few of these methods look like exactly what you'd call in C code, courtesy of a library called <code>[rb-sys](https://github.com/ianks/rb-sys)</code>. The key here is in the name of the method - <code>pub_reverse</code> reverses strings. Here's where the reversal actually happens:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-009-rs">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-009-rs</a></p> <p>There's also an initialization function, <code>Init_rust_ruby_example</code>, to actually define the Ruby modules and methods. Let's piece together what it's doing. Here are the relevant lines for declaring a Ruby module:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-010-rs">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-010-rs</a></p> <p>...and the rest is all adding the <code>reverse</code> method to the module:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-011-rs">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-011-rs</a></p> <p>Note that it needs to translate everything into Matz's Ruby compatible data structures. That includes the module, the module function, and even the string name for the function.</p> <p>(🙋‍♂️ RFC: what is the purpose of <code>[std::mem::tr](transmute)[an](https://doc.rust-lang.org/stable/std/mem/fn.transmute.html)[smute](transmute)</code> here?)</p> <p>💡 <strong>Click</strong> to read more about the purpose of <code>std::mem::transmute</code></p> <hr /> <p>From <a rel="external" href="https://briankung.dev/2022/02/02/writing-a-rust-gem-from-scratch/#comment-274">a comment I wrote on the follow up post:</a></p> <p>After staring at the C header file where <a rel="external" href="https://github.com/ruby/ruby/blob/cb4e2cb55a59833fc4f1f6db2f3082d1ffcafc80/include/ruby/internal/method.h#L28-L99"><code>rb_define_module_function</code> is defined</a> - I don't know C 😰 - I think it's necessary because Rust won't let you pass a function pointer with arbitrary arity, but the C code just assumes that you can. Note that the last argument in <code>rb_define_module_function</code> is an arity indicator. So the transmutation is just ceremony to get a function pointer - any function pointer - past Rust's type system. That's my guess, anyway.</p> <p>EDIT - Seems right! <a rel="external" href="https://twitter.com/_ianks/status/1489419634168184834">https://twitter.com/_ianks/status/1489419634168184834</a></p> <hr /> <h3 id="trying-out-rust-ruby-example">Trying out <code>rust_ruby_example</code></h3> <p>Alright, if you've seen this message:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-012-sh">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-012-sh</a></p> <p>...you're ready to go! Fire up IRB and require <code>rust_ruby_example</code> to take it for a test drive:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-013-sh">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-013-sh</a></p> <p>It reverses the string, as promised. It works!</p> <p>...or does it? Let's see if it's really doing our bidding by modifying the code.</p> <h3 id="adding-a-lowercase-method">Adding a <code>lowercase</code> method</h3> <p>Let's add a <code>RustRubyExample#lowercase</code> method. It will be exactly the same as <code>RustRubyExample#reverse</code>, except it converts case-convertible text to lower case.</p> <p>It should work like this:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-014-sh">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-014-sh</a></p> <p>And we can confirm that it currently does not work:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-015-sh">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-015-sh</a></p> <p>So let's add it. Once again we need <code>#[no_mangle]</code> to tell the compiler not to alter the name of the function once it's been compiled. Mangling essentially namespaces function names so there are no name collisions in the final binary. However, in our case, we want to be able to refer to it by the name we give it in C Ruby, so we don't want our function name to be mangled</p> <p>Add this block of code between the <code>pub_reverse</code> and <code>Init_rust_ruby_example</code> functions in <code>src/lib.rs</code>:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-016-rs">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-016-rs</a></p> <p>We're also going to copy the function signature:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-017-rs">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-017-rs</a></p> <p>(🙋‍♂️ RFC: why does this need the <code>_klass</code> argument?)</p> <p>💡 <strong>Click</strong> to read more about <code>_klass</code> argument</p> <hr /> <p>After figuring out <a href="https://briankung.dev/2022/01/31/sneak-preview-writing-ruby-gem-native-extensions-in-rust/#purpose-of-transmute">the purpose of the call to <code>std::mem::transmute</code></a>, the <code>_klass</code> argument isn't too confusing. It has a leading underscore because it's unused, but the type of functions with that arity on the C side requires a receiver object for the method, even if it goes unused.</p> <hr /> <p>Next we take the Ruby <code>VALUE</code> input and cast it to a Rust string, then lowercase it using standard Rust <code>String</code> methods:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-018-rs">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-018-rs</a></p> <p>...and the rest is all glue code to convert it to a C string and then to a Ruby string:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-019-rs">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-019-rs</a></p> <p>We also need to add the method to the Ruby module. We can do that by duplicating the relevant code in <code>Init_rust_ruby_example</code>:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-020-rs">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-020-rs</a></p> <p>Now to build and reinstall it:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-021-sh">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-021-sh</a></p> <p>Finally, let's test our new functionality:</p> <p><a rel="external" href="https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-022-sh">https://gist.github.com/briankung/d250368e9552dc730c6608727556197d#file-022-sh</a></p> <p>Awesome!</p> <h2 id="conclusion">Conclusion</h2> <p>This is all possible due to Ian's long, hard slog to get this into <code>rubygems</code> proper: <a rel="external" href="https://github.com/rubygems/rubygems/pull/5175">https://github.com/rubygems/rubygems/pull/5175</a>. This could be a hugely impactful addition to <code>rubygems</code> that's been stewing since 2019 and it's mostly been Ian's efforts to get it there. Thanks, Ian!</p> <p><em>If you got this far, check out the follow-up post: <a rel="external" href="https://briankung.dev/2022/02/02/writing-a-rust-gem-from-scratch/">Writing a Rust gem from scratch</a></em></p> 2022-01-07T17:42:56+00:00 2022-01-07T17:42:56+00:00 Unknown https://briankung.dev/2022/01/07/crate-of-the-week-second-chance-list/ <p><strong>TLDR</strong> - the three most upvoted crates that were never recognized as Crate of the Week in <a rel="external" href="https://this-week-in-rust.org">This Week in Rust</a> are <a rel="external" href="https://github.com/tomaka/glutin">glutin</a>, <a rel="external" href="https://github.com/rdambrosio016/rust-cuda">rust-cuda</a>, and <a rel="external" href="https://github.com/tomaka/redshirt">redshirt</a>.</p> <hr /> <p>The Crate of the Week is a Rust (programming language, not game) segment in the This Week in Rust newsletter that highlights a single Rust library in a given week. This helps Rust community members discover and promote crates. You can submit a crate suggestion as well as vote on crates in this <a rel="external" href="https://users.rust-lang.org/t/crate-of-the-week/2704">forum thread</a> by "liking" comments.</p> <p>I was curious whether there were any crates that had never made it onto <a rel="external" href="https://this-week-in-rust.org/">this-week-in-rust.org</a> as a Crate of the Week, despite having a lot of upvotes, so I decided to take a look. I figured it would take an afternoon or so. Little did I know that it would take <em>many</em> afternoons for this busy stay-at-home-dad.</p> <h2 id="methodology">Methodology</h2> <p>I downloaded all of the comments in the TWIR crate of the week thread as JSON via this url:</p> <p><code>[https://users.rust-lang.org/t/crate-of-the-week/2704.json?print=true](https://users.rust-lang.org/t/crate-of-the-week/2704.json?print=true)</code></p> <p>Note the <code>?print=true</code> parameter - this, combined with a <code>&amp;page=</code> parameter lets you page through 1,000 posts in a topic thread at a time.</p> <p>I then associated each url with the number of likes it had received via its parent <code>Post</code> and sorted them by likes in descending order.</p> <p>Originally, I had meant to only compare links submitted within the same week, but eventually decided it just made more sense to look for the most popular links overall. Furthermore, I originally limited the urls to the single most clicked link in a given post, so one post would result in one link (if it had any). I regret both of those decisions, but oh well ¯\_(ツ)_/¯</p> <p>I then began the arduous process of clearing out crates of the week from the resulting list by searching the <code>this-week-in-rust/content</code> folder with ripgrep, then decided to automate it with a quick Ruby script, which also turned out to be more difficult than I bargained for. Trying to recreate my Ruby code in Rust only led to more hair tearing so at this point, I'm satisfied with having manually cleared out nominated crate URLs for crate URLs with 5 or more likes.</p> <p>I now have a much healthier respect for data munging. I might get back to it, but I need a break from this "simple" task for a little bit 😵‍💫</p> <h2 id="results">Results</h2> <p>...for the top 25 crates, anyway. The full results can be found here: <a rel="external" href="https://raw.githubusercontent.com/briankung/cotw_second_chance/main/output.csv">https://raw.githubusercontent.com/briankung/cotw_second_chance/main/output.csv</a></p> <table><thead><tr><th>likes</th><th>url</th></tr></thead><tbody> <tr><td>13</td><td><a rel="external" href="https://github.com/tomaka/glutin">https://github.com/tomaka/glutin</a></td></tr> <tr><td>12</td><td><a rel="external" href="https://github.com/tomaka/redshirt">https://github.com/tomaka/redshirt</a></td></tr> <tr><td>12</td><td><a rel="external" href="https://github.com/rdambrosio016/rust-cuda">https://github.com/rdambrosio016/rust-cuda</a></td></tr> <tr><td>12</td><td><a rel="external" href="https://github.com/kornelski/deunicode">https://github.com/kornelski/deunicode</a></td></tr> <tr><td>12</td><td><a rel="external" href="https://github.com/chowdhurya/rust-unidecode">https://github.com/chowdhurya/rust-unidecode</a></td></tr> <tr><td>11</td><td><a rel="external" href="https://github.com/servo/rust-smallvec">https://github.com/servo/rust-smallvec</a></td></tr> <tr><td>11</td><td><a rel="external" href="https://crates.io/crates/exonum">https://crates.io/crates/exonum</a></td></tr> <tr><td>10</td><td><a rel="external" href="https://lib.rs/crates/async-stream">https://lib.rs/crates/async-stream</a></td></tr> <tr><td>10</td><td><a rel="external" href="https://github.com/matthiaskrgr/cargo-cache">https://github.com/matthiaskrgr/cargo-cache</a></td></tr> <tr><td>9</td><td><a rel="external" href="https://github.com/xiph/rnnoise">https://github.com/xiph/rnnoise</a></td></tr> <tr><td>9</td><td><a rel="external" href="https://github.com/bodil/im-rs">https://github.com/bodil/im-rs</a></td></tr> <tr><td>9</td><td><a rel="external" href="https://crates.io/crates/web-view">https://crates.io/crates/web-view</a></td></tr> <tr><td>8</td><td><a rel="external" href="https://github.com/rustsec/advisory-db">https://github.com/rustsec/advisory-db</a></td></tr> <tr><td>8</td><td><a rel="external" href="https://github.com/rustaudio/dsp-chain">https://github.com/rustaudio/dsp-chain</a></td></tr> <tr><td>8</td><td><a rel="external" href="https://github.com/rustaudio/">https://github.com/rustaudio/</a></td></tr> <tr><td>8</td><td><a rel="external" href="https://github.com/bminixhofer/tractjs">https://github.com/bminixhofer/tractjs</a></td></tr> <tr><td>8</td><td><a rel="external" href="https://github.com/bluss/petulant-avenger-graphlibrary">https://github.com/bluss/petulant-avenger-graphlibrary</a></td></tr> <tr><td>8</td><td><a rel="external" href="https://crates.io/crates/shrinkwraprs">https://crates.io/crates/shrinkwraprs</a></td></tr> <tr><td>8</td><td><a rel="external" href="https://crates.io/crates/duct">https://crates.io/crates/duct</a></td></tr> <tr><td>7</td><td><a rel="external" href="https://lib.rs/crates/enum-unitary">https://lib.rs/crates/enum-unitary</a></td></tr> <tr><td>7</td><td><a rel="external" href="https://github.com/theseus-os/theseus">https://github.com/theseus-os/theseus</a></td></tr> <tr><td>7</td><td><a rel="external" href="https://github.com/sunfishcode/cap-std">https://github.com/sunfishcode/cap-std</a></td></tr> <tr><td>7</td><td><a rel="external" href="https://github.com/sof3/include-flate">https://github.com/sof3/include-flate</a></td></tr> <tr><td>7</td><td><a rel="external" href="https://github.com/rhysd/cargo-husky">https://github.com/rhysd/cargo-husky</a></td></tr> <tr><td>7</td><td><a rel="external" href="https://github.com/matklad/once_cell/">https://github.com/matklad/once_cell/</a></td></tr> </tbody></table> <p>You can view my attempt to use Rust as a scripting language here: <a rel="external" href="https://github.com/briankung/cotw_second_chance">https://github.com/briankung/cotw_second_chance</a></p> <h2 id="conclusion">Conclusion</h2> <p>I initially decided to do this entirely in Rust as much as possible because it seemed like a small task suitable for learning. For instance, instead of downloading the JSON separately, I decided to download the file in Rust using tokio and reqwest. That actually went off pretty much without a hitch.</p> <p>Then I got a bit sidetracked figuring out that I didn't have to declare every field in my JSON structure before I deserialized it with serde - though I did find the fantastic <a rel="external" href="https://transform.tools/json-to-rust-serde">transform.tools</a> website to help with that. Also serde and its documentation seemed really opaque to me at first - was I supposed to implement <code>Deserialize</code>, and if so, then how? Finally, I pointed the right <code>serde_json</code> method at the JSON data and carried on. I originally wanted to make sure that the posts were in order and grouped by week, but I found the <code>time</code> crate to be similarly difficult to find examples for, and a <a rel="external" href="https://time-rs.github.io/book/">bewilderingly empty book</a> left me delaying the <code>time</code> related features. Ironically, I didn't group the posts by week in the end, anyway, so I could have ignored the timestamps.</p> <p>And finally when I thought I had gotten something halfway acceptable, I still found a good deal of crates that had already been CotW winners in my filtered list of crates. "How many incorrect entries could there be?" I wondered naively, and then began to search <code>"crate_name site:this-week-in-rust.org"</code> on Google and discovered that the answer to my question was "quite a lot, actually." I ended up using ripgrep in the command line to search the <code>this-week-in-rust</code> repo, then got frustrated and jumped ship back to Ruby to automate it. After writing a script in my native tongue of Ruby, this is how I felt:</p> <figure> <p><a href="/assets/images/60iq8m.jpg"><img src="/assets/images/60iq8m.jpg" alt="" /></a></p> <figcaption> <p>Update - I rewrote the logic in Rust, after all.</p> </figcaption> </figure> <p>However, the biggest problem was that I underestimated how difficult it would be to beat the data I had into the proper shape. As it turns out, data munging is not a simple task. Comments aren't uniform, URLs aren't uniform, and even issues of This Week in Rust aren't uniform. At one point I was searching for a markdown header like <code>"# Crate of the Week"</code> and discovered that one issue had it as <code>"# Crate &amp; Quote of the Week"</code>.</p> <p>In the end, I did finally use a lot of libraries I'd been wanting to use and got a hang of a bit more corners of the Rust world, so it was still a worthwhile endeavor. While there must be smarter ways to handle this problem, I've run out of enthusiasm for it, so I'm putting it on the backburner. You're welcome to take a crack at it yourself! I'd love to hear about it if you do.</p> <p>Oh, and...don't forget to nominate crates in the <a rel="external" href="https://users.rust-lang.org/t/crate-of-the-week/2704">Crate of the Week thread!</a></p> 2021-12-07T07:22:04+00:00 2021-12-07T07:22:04+00:00 Unknown https://briankung.dev/2021/12/07/building-a-cedict-parser-in-rust-with-nom/ <p>The <a rel="external" href="https://cc-cedict.org/wiki/format:syntax">CEDICT format</a> is a simple, creative commons-licensed file format for Chinese/English dictionaries. While Mandarin-only CEDICT parsers abound, there is basically no support for Cantonese jyutping in the English-speaking programming world. As someone who would have liked to use Cantonese pronunciations in my programs, at first I was stuck. I considered adding jyutping support to an existing parser, but the example parser I found seemed difficult to extend in such a way. I had also never hand-written a parser before, other than some very simple tutorials.</p> <p>I forgot how I stumbled across it, but that's when I found <a rel="external" href="https://docs.rs/nom/latest/nom/">nom</a>, a "<a rel="external" href="https://en.wikipedia.org/wiki/Parser_combinator">parser combinator</a>" library whose motto is "eating data byte by byte." Reading the documentation, I found that I could actually understand <a rel="external" href="https://docs.rs/nom/latest/nom/#example">the example code</a> for parsing a hexadecimal number into a color code.</p> <figure> <p><a href="/assets/images/nom.png"><img src="/assets/images/nom.png" alt="" /></a></p> <figcaption> <p><em>nom, eating data byte by byte</em></p> </figcaption> </figure> <p>What are parser combinators? I had never really looked into the term before this, assuming, as usual with topics in programming, that it was above me (this is the same cursed reason I avoided doing computer science in college). It turns out that they are a way to construct parsers like Lego blocks, piece by piece, until you have pieces that have the semantic meanings that you want, and can manipulate as you please.</p> <p>The basic building blocks of parser combinators are as simple as commanding the computer to recognize (and separate) alphabetic characters, white spaces, or numerical characters. They get more complicated when you combine them, e.g. in English you would say "take as many alphabetic characters as possible then a number, and we'll call that a <code>syllable</code>. In this spot between the brackets, there may be many <code>syllables</code> so take as many <code>syllables</code> as you can until you encounter the end bracket."</p> <p>After <code>nom</code> faithfully eats and sorts bytes for me, I get to store them into meaningful data structures.</p> <p>Enough pseudo code, here's the Rust code (using <code>nom 6.1.2</code>) I used to parse the CEDICT entries. We're looking to parse the following format:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>你好嗎 你好吗 [ni3 hao3 ma5] {nei5 hou2 maa1} /how are you?/ # CEDICT also supports comments</span></span></code></pre> <p>There are 5 sections delimited by spaces:</p> <ul> <li> <p>Traditional Chinese characters</p> </li> <li> <p>Simplified Chinese characters</p> </li> <li> <p>Pinyin, itself delimited by square brackets</p> </li> <li> <p>Jyutping, delimited by squiggly brackets, an addition to the CEDICT format by <a rel="external" href="https://cantonese.org/">Cantonese.org</a></p> </li> <li> <p>English language definitions delimited by forward slashes</p> </li> <li> <p>…and comments, though I'm unsure if they're supported at the end of the line as written. I'll skip the comments for now.</p> </li> </ul> <p>While parser combinators are easy to write from bottom to top, it may be easiest to understand them top down - I'll start at the highest level of abstraction and gradually drill down into the details. Another bonus of parser combinators - it's a joy to read them afterward, even after a few months!</p> <p>So here's the parser for a <code>CedictEntry</code>, the Rust struct I'm using to store data about a single entry's various parts:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #8839EF;">fn</span><span style="color: #1E66F5;font-style: italic;"> cedict_entry</span><span style="color: #7C7F93;">(</span><span style="color: #E64553;">i</span><span style="color: #179299;">: &amp;</span><span style="color: #DF8E1D;font-style: italic;">str</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> -&gt;</span><span style="color: #DF8E1D;font-style: italic;"> IResult</span><span style="color: #04A5E5;">&lt;</span><span style="color: #179299;">&amp;</span><span style="color: #DF8E1D;font-style: italic;">str</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> CedictEntry</span><span style="color: #04A5E5;">&gt;</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #8839EF;"> let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> traditional</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> not_whitespace</span><span style="color: #7C7F93;">(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> _</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #DF8E1D;"> character</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">space1</span><span style="color: #7C7F93;">(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> simplified</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> not_whitespace</span><span style="color: #7C7F93;">(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> _</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #DF8E1D;"> character</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">space1</span><span style="color: #7C7F93;">(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> pinyin</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> pinyin</span><span style="color: #7C7F93;">(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> _</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #DF8E1D;"> character</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">space1</span><span style="color: #7C7F93;">(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> jyutping</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #DF8E1D;"> combinator</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">opt</span><span style="color: #7C7F93;">(</span><span>jyutping</span><span style="color: #7C7F93;">)(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> _</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #DF8E1D;"> character</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">space0</span><span style="color: #7C7F93;">(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> definitions</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> definitions</span><span style="color: #7C7F93;">(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Ok</span><span style="color: #7C7F93;">((</span></span> <span class="giallo-l"><span> i</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> CedictEntry</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span> traditional</span><span style="color: #179299;">:</span><span> traditional</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">into</span><span style="color: #7C7F93;">(),</span></span> <span class="giallo-l"><span> simplified</span><span style="color: #179299;">:</span><span> simplified</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">into</span><span style="color: #7C7F93;">(),</span></span> <span class="giallo-l"><span> pinyin</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span> jyutping</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span> definitions</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> },</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> ))</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span></code></pre> <p>Let's start with the function signature:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #8839EF;">fn</span><span style="color: #1E66F5;font-style: italic;"> cedict_entry</span><span style="color: #7C7F93;">(</span><span style="color: #E64553;">i</span><span style="color: #179299;">: &amp;</span><span style="color: #DF8E1D;font-style: italic;">str</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> -&gt;</span><span style="color: #DF8E1D;font-style: italic;"> IResult</span><span style="color: #04A5E5;">&lt;</span><span style="color: #179299;">&amp;</span><span style="color: #DF8E1D;font-style: italic;">str</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> CedictEntry</span><span style="color: #04A5E5;">&gt;</span></span></code></pre> <p>This is a function definition in Rust declaring a function <code>cedict_entry</code> that takes an input <code>i</code> of type <code>&amp;str</code> and returns nom's <code>IResult</code> type, which itself contains <code>&amp;str</code> and a <code>CedictEntry</code> if all goes well. If all doesn't go well, it will return an error, which you can see in <a rel="external" href="https://docs.rs/nom/latest/nom/type.IResult.html"><code>IResult</code>'s type definition</a>. The <code>&amp;str</code> types here are important because nom attempts to be zero-copy for performance reasons - the <code>&amp;str</code> type is immutable and does not allocate memory, serving only as a read-only window into data already loaded into memory.</p> <p>The main body of the function is <em>almost</em> readable as pseudo-code:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #8839EF;">let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> traditional</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> not_whitespace</span><span style="color: #7C7F93;">(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;">let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> _</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #DF8E1D;"> character</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">space1</span><span style="color: #7C7F93;">(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;">let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> simplified</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> not_whitespace</span><span style="color: #7C7F93;">(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;">let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> _</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #DF8E1D;"> character</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">space1</span><span style="color: #7C7F93;">(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;">let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> pinyin</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> pinyin</span><span style="color: #7C7F93;">(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;">let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> _</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #DF8E1D;"> character</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">space1</span><span style="color: #7C7F93;">(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;">let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> jyutping</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #DF8E1D;"> combinator</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">opt</span><span style="color: #7C7F93;">(</span><span>jyutping</span><span style="color: #7C7F93;">)(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;">let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> _</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #DF8E1D;"> character</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">space0</span><span style="color: #7C7F93;">(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;">let</span><span style="color: #7C7F93;"> (</span><span>i</span><span style="color: #7C7F93;">,</span><span> definitions</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> definitions</span><span style="color: #7C7F93;">(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span></code></pre> <p>This takes advantage of a few Rust features that come together nicely. First, <code>let</code> statements can do destructuring assignment. So if a function returns a tuple <code>(1, 2)</code>, you can <code>let (a, b) = (1, 2)</code> and <code>a</code> will be equal to <code>1</code> and <code>b</code> will be equal to <code>2</code>.</p> <p>Second, Rust emphasizes safety and these parsers may fail, so we have to handle it. In Rust, expected failures are captured in the type system, specifically the <code>Result</code> type, which allow you to either extract the value or handle the error without the program blowing up and panicking. Parsers in nom return an <code>IResult</code>, which is a custom <code>Result</code> type. When the parsers fail, nom will default to returning a <code>nom::ErrorKind</code>. When they succeed, nom parsers will return a tuple of two elements, the first being the remaining input and the second being the result of the parsing. So if you were parsing <code>"abc"</code> with a parser combinator that looked for the character <code>"a"</code>, for example, it might look something like this:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #8839EF;">let</span><span> letter_a_parser</span><span style="color: #179299;"> =</span><span style="color: #DF8E1D;"> bytes</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">tag</span><span style="color: #7C7F93;">(</span><span style="color: #40A02B;">&quot;a&quot;</span><span style="color: #7C7F93;">);</span></span> <span class="giallo-l"><span style="color: #8839EF;">let</span><span> result</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> letter_a_parser</span><span style="color: #7C7F93;">(</span><span style="color: #40A02B;">&quot;abc&quot;</span><span style="color: #7C7F93;">);</span></span> <span class="giallo-l"><span># result</span><span style="color: #179299;"> ==</span><span style="color: #7C7F93;"> (</span><span style="color: #40A02B;">&quot;bc&quot;</span><span style="color: #7C7F93;">,</span><span style="color: #40A02B;"> &quot;a&quot;</span><span style="color: #7C7F93;">)</span></span></code></pre> <p>But what happens in this code when one of these function calls returns an error? Is it just stored in the variables declared in the <code>let</code> statement? That's where the question mark at the end of the line comes into play, Rust's third salient feature. The question mark is a shortcut for saying, "if the Result of an expression is a valid value, the expression evaluates as that value. Otherwise if the Result is an error, then have not just the expression, but the entire calling function return the error."</p> <p>That way we can have our <code>let</code> tuples and our errors, too. 🍰</p> <p>Finally, we have the code that actually stores the captured data in a struct:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;">Ok</span><span style="color: #7C7F93;">((</span></span> <span class="giallo-l"><span> i</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> CedictEntry</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span> traditional</span><span style="color: #179299;">:</span><span> traditional</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">into</span><span style="color: #7C7F93;">(),</span></span> <span class="giallo-l"><span> simplified</span><span style="color: #179299;">:</span><span> simplified</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">into</span><span style="color: #7C7F93;">(),</span></span> <span class="giallo-l"><span> pinyin</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span> jyutping</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span> definitions</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> },</span></span> <span class="giallo-l"><span style="color: #7C7F93;">))</span></span></code></pre> <p><code>Ok</code> is a variant of the <code>Result</code> <a rel="external" href="http://web.mit.edu/rust-lang_v1.25/arch/amd64_ubuntu1404/share/doc/rust/html/book/first-edition/enums.html">enum</a>, indicating that the function executed properly. The function returns an <code>Ok</code> containing the rest of the input, <code>i</code>, so that the next parser can have a go at the input, and a <code>CedictEntry</code> representing, well, the parsed Cedict entry. If anything <em>does</em> go wrong, it will return an <code>Err</code>, which will need to be handled at the site of the function call.</p> <p>Now, a few of the parsers I used in the body of the function were provided by nom: <code>not_whitespace</code>, <code>character::complete::space1</code>, and <code>character::complete::space0</code>, for instance. As mentioned before, <code>not_whitespace</code> will match any character that isn't whitespace, whereas <code>space1</code> will match a single space and <code>space0</code> will match zero or one space.</p> <p>But a few of these are custom parsers that I wrote from nom's more basic building blocks: <code>pinyin</code>, <code>jyutping</code>, and <code>definitions</code>. Let's take a look at those.</p> <h2 id="the-pinyin-jyutping-parsers">The pinyin / jyutping parsers</h2> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>1️⃣ 你好嗎 2️⃣ 你好吗 3️⃣ [ni3 hao3 ma5] 4️⃣ {nei5 hou2 maa1} 5️⃣ /how are you?/</span></span></code></pre> <p>We managed to get by using nom's builtin <code>not_whitespace</code> parser for 1️⃣ and 2️⃣ (as labeled above). Basically, we expect the start of the line to be characters, then a space, then more characters of a different sort. However, we can't simply limit it to Han Unicode characters because some of the characters in the dictionary involve alphanumeric characters. In the interests of simplicity, I opted for a parser that simply matches any non-whitespace character for the first two.</p> <p>For number 3️⃣, however, we're going to need to do things a little differently. I'd like to be able to retain specific information about each syllable instead of just grabbing the entire string like I did with the Chinese characters, so the parser is more complicated. Here's the parser for <code>pinyin</code>:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #8839EF;">fn</span><span style="color: #1E66F5;font-style: italic;"> pinyin</span><span style="color: #7C7F93;">(</span><span style="color: #E64553;">i</span><span style="color: #179299;">: &amp;</span><span style="color: #DF8E1D;font-style: italic;">str</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> -&gt;</span><span style="color: #DF8E1D;font-style: italic;"> IResult</span><span style="color: #04A5E5;">&lt;</span><span style="color: #179299;">&amp;</span><span style="color: #DF8E1D;font-style: italic;">str</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> Option</span><span style="color: #04A5E5;">&lt;</span><span style="color: #DF8E1D;font-style: italic;">Vec</span><span style="color: #04A5E5;">&lt;</span><span style="color: #DF8E1D;font-style: italic;">Syllable</span><span style="color: #04A5E5;">&gt;&gt;&gt;</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span> # </span><span style="color: #DF8E1D;font-style: italic;">Here</span><span> we split the pronunciations out from their square bracket delimiters</span><span style="color: #179299;">:</span></span> <span class="giallo-l"><span style="color: #8839EF;"> let</span><span style="color: #7C7F93;"> (</span><span>rest</span><span style="color: #7C7F93;">, (</span><span>_</span><span style="color: #7C7F93;">,</span><span> pronunciations</span><span style="color: #7C7F93;">,</span><span> _</span><span style="color: #7C7F93;">))</span><span style="color: #179299;"> =</span><span style="color: #DF8E1D;"> sequence</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">tuple</span><span style="color: #7C7F93;">((</span></span> <span class="giallo-l"><span style="color: #DF8E1D;"> bytes</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">tag</span><span style="color: #7C7F93;">(</span><span style="color: #40A02B;">&quot;[&quot;</span><span style="color: #7C7F93;">),</span></span> <span class="giallo-l"><span style="color: #DF8E1D;"> combinator</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">opt</span><span style="color: #7C7F93;">(</span><span style="color: #DF8E1D;">bytes</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">is_not</span><span style="color: #7C7F93;">(</span><span style="color: #40A02B;">&quot;]&quot;</span><span style="color: #7C7F93;">)),</span></span> <span class="giallo-l"><span style="color: #DF8E1D;"> bytes</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">tag</span><span style="color: #7C7F93;">(</span><span style="color: #40A02B;">&quot;]&quot;</span><span style="color: #7C7F93;">),</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> ))(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span> # </span><span style="color: #DF8E1D;font-style: italic;">Here</span><span> we handle the case</span><span style="color: #8839EF;"> where</span><span> there</span><span style="color: #7C7F93;">&#39;</span><span style="color: #1E66F5;font-style: italic;">s</span><span> nothing between the brackets</span></span> <span class="giallo-l"><span style="color: #8839EF;"> if let</span><span style="color: #DF8E1D;font-style: italic;"> Some</span><span style="color: #7C7F93;">(</span><span>pronunciations</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span> pronunciations</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #8839EF;"> let</span><span style="color: #7C7F93;"> (</span><span>_</span><span style="color: #7C7F93;">,</span><span> syllables</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> syllables</span><span style="color: #7C7F93;">(</span><span>pronunciations</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Ok</span><span style="color: #7C7F93;">((</span><span>rest</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> Some</span><span style="color: #7C7F93;">(</span><span>syllables</span><span style="color: #7C7F93;">)))</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> }</span><span style="color: #8839EF;"> else</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Ok</span><span style="color: #7C7F93;">((</span><span>rest</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> None</span><span style="color: #7C7F93;">))</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> }</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span></code></pre> <p>Let's talk about this in detail, starting again with the function signature:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #8839EF;">fn</span><span style="color: #1E66F5;font-style: italic;"> pinyin</span><span style="color: #7C7F93;">(</span><span style="color: #E64553;">i</span><span style="color: #179299;">: &amp;</span><span style="color: #DF8E1D;font-style: italic;">str</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> -&gt;</span><span style="color: #DF8E1D;font-style: italic;"> IResult</span><span style="color: #04A5E5;">&lt;</span><span style="color: #179299;">&amp;</span><span style="color: #DF8E1D;font-style: italic;">str</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> Option</span><span style="color: #04A5E5;">&lt;</span><span style="color: #DF8E1D;font-style: italic;">Vec</span><span style="color: #04A5E5;">&lt;</span><span style="color: #DF8E1D;font-style: italic;">Syllable</span><span style="color: #04A5E5;">&gt;&gt;&gt;</span></span></code></pre> <p>This takes a <code>&amp;str</code> and returns an <code>IResult&lt;&amp;str, Option&lt;Vec&lt;Syllable&gt;&gt;&gt;</code>. The <code>IResult</code> returns a tuple of <code>(&amp;str, Option&lt;Vec&lt;Syllable&gt;&gt;</code>. In the case where we have parsed a valid line such as the one above, <code>&amp;str</code> is a reference to the remainder of the input - in this case, it would be parts 4️⃣ and 5️⃣ as shown below:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="plain"><span class="giallo-l"><span>&quot; {nei5 hou2 maa1} /how are you?/&quot;</span></span></code></pre> <p>And the <code>Option&lt;Vec&lt;Syllable&gt;&gt;</code> resulting from the <code>pinyin</code> parser would look like this:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;">Some</span><span style="color: #7C7F93;">([</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Syllable</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span> pronunciation</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> &quot;ni&quot;</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span> tone</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> &quot;3&quot;</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> },</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Syllable</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span> pronunciation</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> &quot;hao&quot;</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span> tone</span><span style="color: #179299;">:</span><span style="color: #40A02B;"> &quot;3&quot;</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> },</span></span> <span class="giallo-l"><span style="color: #7C7F93;">])</span></span></code></pre> <p><em>(Note that if there were no valid syllables between the brackets it would result in a <code>None</code> return type instead of a <code>Some</code> containing a <code>Vec</code>. Such is the nature of the <code>Option</code> <code>Enum</code>.)</em></p> <p>First, we need to separate the pronunciations from their brackets, which is what this chunk of code does:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #8839EF;">let</span><span> 0️⃣ </span><span style="color: #7C7F93;">(</span><span>rest</span><span style="color: #7C7F93;">, (</span><span>_</span><span style="color: #7C7F93;">,</span><span> pronunciations</span><span style="color: #7C7F93;">,</span><span> _</span><span style="color: #7C7F93;">))</span><span style="color: #179299;"> =</span><span style="color: #DF8E1D;"> sequence</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">tuple</span><span style="color: #7C7F93;">((</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> 1️⃣ </span><span style="color: #DF8E1D;">bytes</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">tag</span><span style="color: #7C7F93;">(</span><span style="color: #40A02B;">&quot;[&quot;</span><span style="color: #7C7F93;">),</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> 2️⃣ </span><span style="color: #DF8E1D;">combinator</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">opt</span><span style="color: #7C7F93;">(</span><span style="color: #DF8E1D;">bytes</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">is_not</span><span style="color: #7C7F93;">(</span><span style="color: #40A02B;">&quot;]&quot;</span><span style="color: #7C7F93;">)),</span></span> <span class="giallo-l"><span style="color: #1E66F5;"> 3️⃣ </span><span style="color: #DF8E1D;">bytes</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">tag</span><span style="color: #7C7F93;">(</span><span style="color: #40A02B;">&quot;]&quot;</span><span style="color: #7C7F93;">),</span></span> <span class="giallo-l"><span style="color: #7C7F93;">))(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span></code></pre> <p>I'm using <code>sequence::tuple</code>, which is a built-in nom parser that takes a tuple of parser combinators and returns a tuple of the rest of the output and an inner tuple of all of the components we ask for. The tuple we supplied included:</p> <ul> <li> <p>1️⃣ a parser for the opening bracket of the pinyin,</p> </li> <li> <p>2️⃣ a parser for any bytes that are not equal to <code>"]"</code>, made optional via nom's built-in parser, <code>combinator::opt</code>, and</p> </li> <li> <p>3️⃣ the ending bracket.</p> </li> </ul> <p>We expect the output in the case of a successful parse to contain the rest of the input string, 0️⃣, followed by a tuple of <code>(1️⃣, 2️⃣, and 3️⃣)</code>, of which we only retain 2️⃣. In Rust, any variable beginning with an underscore tells the compiler we won't be using it, and it is a compile time warning if either an underscored variable <em>is</em> used, or if a non-underscored variable goes _un_used.</p> <p>By the end of this, we should have two variables declared: <code>rest</code>, which is the <a href="https://briankung.dev/2021/12/07/building-a-cedict-parser-in-rust-with-nom/#rest-of-the-input">rest of the input</a>, and <code>pronunciations</code>, which is whatever was contained between the two square brackets - in this case <code>"ni3 hao3 ma5"</code>. Now to extract the individual syllables.</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #8839EF;">if let</span><span style="color: #DF8E1D;font-style: italic;"> Some</span><span style="color: #7C7F93;">(</span><span>pronunciations</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span> pronunciations</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #8839EF;"> let</span><span style="color: #7C7F93;"> (</span><span>_</span><span style="color: #7C7F93;">,</span><span> syllables</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> syllables</span><span style="color: #7C7F93;">(</span><span>pronunciations</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Ok</span><span style="color: #7C7F93;">((</span><span>rest</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> Some</span><span style="color: #7C7F93;">(</span><span>syllables</span><span style="color: #7C7F93;">)))</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span><span style="color: #8839EF;"> else</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Ok</span><span style="color: #7C7F93;">((</span><span>rest</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> None</span><span style="color: #7C7F93;">))</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span></code></pre> <p>We then check if pronunciations is a <code>None</code> type indicating that it didn't find what you were looking for, or a <code>Some</code> type which includes the combinator results you asked for. If the result is <code>Some(pronunciations)</code> then we punt it down the line to the <code>syllables</code> parser combinator:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #7C7F93;font-style: italic;">/// takes a series of possibly undelimited syllables such as &quot;ni3hao3&quot; and returns a Vec of Syllables</span></span> <span class="giallo-l"><span style="color: #8839EF;">fn</span><span style="color: #1E66F5;font-style: italic;"> syllables</span><span style="color: #7C7F93;">(</span><span style="color: #E64553;">i</span><span style="color: #179299;">: &amp;</span><span style="color: #DF8E1D;font-style: italic;">str</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> -&gt;</span><span style="color: #DF8E1D;font-style: italic;"> IResult</span><span style="color: #04A5E5;">&lt;</span><span style="color: #179299;">&amp;</span><span style="color: #DF8E1D;font-style: italic;">str</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> Vec</span><span style="color: #04A5E5;">&lt;</span><span style="color: #DF8E1D;font-style: italic;">Syllable</span><span style="color: #04A5E5;">&gt;&gt;</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #DF8E1D;"> multi</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">many0</span><span style="color: #7C7F93;">(</span><span>syllable</span><span style="color: #7C7F93;">)(</span><span>i</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span> <span class="giallo-l"><span style="color: #8839EF;">fn</span><span style="color: #1E66F5;font-style: italic;"> syllable</span><span style="color: #7C7F93;">(</span><span style="color: #E64553;">i</span><span style="color: #179299;">: &amp;</span><span style="color: #DF8E1D;font-style: italic;">str</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> -&gt;</span><span style="color: #DF8E1D;font-style: italic;"> IResult</span><span style="color: #04A5E5;">&lt;</span><span style="color: #179299;">&amp;</span><span style="color: #DF8E1D;font-style: italic;">str</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> Syllable</span><span style="color: #04A5E5;">&gt;</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #8839EF;"> let</span><span style="color: #7C7F93;"> (</span><span>rest</span><span style="color: #7C7F93;">, (</span><span>_</span><span style="color: #7C7F93;">,</span><span> pronunciation</span><span style="color: #7C7F93;">,</span><span> tone</span><span style="color: #7C7F93;">))</span><span style="color: #179299;"> =</span><span style="color: #DF8E1D;"> sequence</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">tuple</span><span style="color: #7C7F93;">((</span></span> <span class="giallo-l"><span style="color: #DF8E1D;"> character</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span>space0</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #DF8E1D;"> character</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span>alpha1</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #DF8E1D;"> character</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span>digit0</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> ))(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Ok</span><span style="color: #7C7F93;">((</span><span>rest</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> Syllable</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">new</span><span style="color: #7C7F93;">(</span><span>pronunciation</span><span style="color: #7C7F93;">,</span><span> tone</span><span style="color: #7C7F93;">)))</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span></code></pre> <p>Here, I've broken syllables down into two separate functions, but we could easily inline the <code>syllable</code> parser. We start out with <code>multi::many0</code> which will match zero or more <code>syllable</code> parsers. Within the <code>syllable</code> parser we can see another <code>sequence::tuple</code> parser, which we are supplying with <code>space0</code>, <code>alpha1</code>, and <code>digit0</code>. Translated to English, this would match against zero or more spaces, then one or more alphabetic ASCII characters, then zero or more ASCII numerical characters, representing any leading whitespace, the pronunciation, and finally the tone (see more about tones in Mandarin <a rel="external" href="https://en.wikipedia.org/wiki/Standard_Chinese_phonology#Tones">here</a>).</p> <p>If that's fine, we return the rest of the input to the <code>multi::many0</code> parser, as well as a new <code>Syllable</code> struct populated with the pronunciation and the tone, then do it over and over again until the <code>syllable</code> parser stops matching. Finally, we're left with our <code>Vec</code> of <code>Syllable</code> structs, as captured in this unit test:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;">#</span><span style="color: #7C7F93;">[</span><span style="color: #DF8E1D;font-style: italic;">test</span><span style="color: #7C7F93;">]</span></span> <span class="giallo-l"><span style="color: #8839EF;">fn</span><span style="color: #1E66F5;font-style: italic;"> test_parse_syllables</span><span style="color: #7C7F93;">() {</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;"> assert_eq!</span><span style="color: #7C7F93;">(</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;"> syllables</span><span style="color: #7C7F93;">(</span><span style="color: #40A02B;">&quot;ni3hao3&quot;</span><span style="color: #7C7F93;">),</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Ok</span><span style="color: #7C7F93;">((</span></span> <span class="giallo-l"><span style="color: #40A02B;"> &quot;&quot;</span><span style="color: #7C7F93;">,</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;"> vec!</span><span style="color: #7C7F93;">[</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Syllable</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">new</span><span style="color: #7C7F93;">(</span><span style="color: #40A02B;">&quot;ni&quot;</span><span style="color: #7C7F93;">,</span><span style="color: #40A02B;"> &quot;3&quot;</span><span style="color: #7C7F93;">),</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Syllable</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">new</span><span style="color: #7C7F93;">(</span><span style="color: #40A02B;">&quot;hao&quot;</span><span style="color: #7C7F93;">,</span><span style="color: #40A02B;"> &quot;3&quot;</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> ]</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> ))</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> );</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span></code></pre> <p>Thankfully, we can do much the same thing to handle the Cantonese jyutping pronunciations, just by changing the delimiters:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #8839EF;">fn</span><span style="color: #1E66F5;font-style: italic;"> jyutping</span><span style="color: #7C7F93;">(</span><span style="color: #E64553;">i</span><span style="color: #179299;">: &amp;</span><span style="color: #DF8E1D;font-style: italic;">str</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> -&gt;</span><span style="color: #DF8E1D;font-style: italic;"> IResult</span><span style="color: #04A5E5;">&lt;</span><span style="color: #179299;">&amp;</span><span style="color: #DF8E1D;font-style: italic;">str</span><span style="color: #7C7F93;">,</span><span style="color: #DF8E1D;font-style: italic;"> Vec</span><span style="color: #04A5E5;">&lt;</span><span style="color: #DF8E1D;font-style: italic;">Syllable</span><span style="color: #04A5E5;">&gt;&gt;</span><span style="color: #7C7F93;"> {</span></span> <span class="giallo-l"><span style="color: #8839EF;"> let</span><span style="color: #7C7F93;"> (</span><span>rest</span><span style="color: #7C7F93;">,</span><span> pronunciations</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #DF8E1D;"> sequence</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">delimited</span><span style="color: #7C7F93;">(</span></span> <span class="giallo-l"><span style="color: #DF8E1D;"> bytes</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">tag</span><span style="color: #7C7F93;">(</span><span style="color: #40A02B;">&quot;{&quot;</span><span style="color: #7C7F93;">),</span></span> <span class="giallo-l"><span style="color: #DF8E1D;"> bytes</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">is_not</span><span style="color: #7C7F93;">(</span><span style="color: #40A02B;">&quot;}&quot;</span><span style="color: #7C7F93;">),</span></span> <span class="giallo-l"><span style="color: #DF8E1D;"> bytes</span><span style="color: #179299;">::</span><span style="color: #DF8E1D;">complete</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">tag</span><span style="color: #7C7F93;">(</span><span style="color: #40A02B;">&quot;}&quot;</span><span style="color: #7C7F93;">),</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> )(</span><span>i</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #8839EF;"> let</span><span style="color: #7C7F93;"> (</span><span>_</span><span style="color: #7C7F93;">,</span><span> syllables</span><span style="color: #7C7F93;">)</span><span style="color: #179299;"> =</span><span style="color: #1E66F5;font-style: italic;"> syllables</span><span style="color: #7C7F93;">(</span><span>pronunciations</span><span style="color: #7C7F93;">)</span><span style="color: #179299;">?</span><span style="color: #7C7F93;">;</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> Ok</span><span style="color: #7C7F93;">((</span><span>rest</span><span style="color: #7C7F93;">,</span><span> syllables</span><span style="color: #7C7F93;">))</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span></code></pre> <p>Hopefully that helps explain this particular use case of nom in Rust. Also, if I’ve made any mistakes in my explanations, please let me know! I’m new both to parsers and to Rust. As a novice to writing parsers, I found it very approachable, if a bit mystifying which parsers to use, at first.</p> <p>On that note, nom divides its parser combinators into a couple major categories. There's more to nom, and I've only just begun to explore it, but my mental map of the combinators are bytes versus characters, complete vs streaming, collections, and control flow combinators. I've mapped them out below:</p> <ul> <li> <p>bytes vs characters - literally represented by <code>nom::bytes</code> vs <code>nom::character</code>. For when you need to parse at the byte level vs characters. The distinction can be a little fuzzy to me.</p> </li> <li> <p>complete vs streaming - the above modules each split into <code>complete</code> and <code>streaming</code> versions for when you have to handle streaming input versus when you have the entire input data at your disposal.</p> </li> <li> <p>collections - the <code>nom::multi</code> module will give you parsers for handling repetitive data. We used this module to handle multiple syllables in pinyin and jyutping.</p> </li> <li> <p>control flow - I think of <code>nom::branch</code> and <code>nom::combinator</code> as being control flow</p> </li> </ul> <p>Sometimes the hardest thing is finding the right parser combinator. For that, there is now a handy guide: <a rel="external" href="https://github.com/Geal/nom/blob/main/doc/choosing_a_combinator.md">https://github.com/Geal/nom/blob/main/doc/choosing_a_combinator.md</a></p> <p>Note that there are many more modules that I haven't used that look like they would come in handy, such as <code>nom::number</code> for parsing numbers and <code>nom::recipes</code>, which might have saved me a good amount of time!</p> <p>Happy parsing!</p> <h2 id="additional-links">Additional Links</h2> <ul> <li> <p><a rel="external" href="https://github.com/Geal/nom">nom on GitHub</a></p> </li> <li> <p><a rel="external" href="https://docs.rs/nom/7.1.0/nom/">nom docs on docs.rs</a></p> </li> <li> <p><a rel="external" href="https://gitter.im/Geal/nom">nom on Gitter.im chat</a></p> </li> <li> <p><a rel="external" href="https://github.com/benkay86/nom-tutorial">benkay86's handy tutorial (nom v5)</a></p> </li> <li> <p><a rel="external" href="https://github.com/briankung/cccedict">briankung/cccedict parser on GitHub</a></p> </li> <li> <p><a rel="external" href="https://cc-cedict.org/wiki/format:syntax">The CC-CEDICT format</a></p> </li> <li> <p><a rel="external" href="https://www.mdbg.net/chinese/dictionary?page=cedict">CC-CEDICT download page</a></p> </li> <li> <p><a rel="external" href="https://cantonese.org/download.html">CC-Canto download page</a></p> </li> </ul> 2021-11-20T22:30:09+00:00 2021-11-20T22:30:09+00:00 Unknown https://briankung.dev/2021/11/20/contributing-to-artichoke-in-rust/ <p>Yesterday, I eked out a bit of time to contribute to Artichoke in Rust, in no small part due to the maintainer <a rel="external" href="https://github.com/lopopolo">@lopopolo</a>:</p> <figure> <p>[](/assets/images/screen-shot-2021-11-20-at-4.45.32-pm.png) <figcaption> <p>@lopopolo was very welcoming!</p> </figcaption> </figure> <p>Artichoke is an implementation of Ruby written in the Rust programming language. The issue I worked on was to expose a newly stabilized method in Artichoke's implementation of the Ruby String class. In Rust, strings are represented as Vectors of <code>u8</code> bytes which are guaranteed to be valid UTF-8 sequences. As of Rust version 1.56.0, the compiler stabilized the <a rel="external" href="https://doc.rust-lang.org/stable/std/vec/struct.Vec.html#method.shrink_to">Vec::shrink_to</a> method, which allows you to reduce the amount of memory a Vec takes, down to a lower bound, which you supply. So if you have a String and it has allocated 4 bytes, but it's only taking up 1 byte, you can <code>string.shrink_to(2)</code> to reduce its size to no less than 2 bytes. Here's that same scenario in code:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #8839EF;">let mut</span><span> string</span><span style="color: #179299;"> =</span><span style="color: #DF8E1D;font-style: italic;"> String</span><span style="color: #179299;">::</span><span style="color: #1E66F5;font-style: italic;">with_capacity</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">4</span><span style="color: #7C7F93;">);</span></span> <span class="giallo-l"><span>string</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">extend</span><span style="color: #7C7F93;">(</span><span style="color: #40A02B;">&quot;1&quot;</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">chars</span><span style="color: #7C7F93;">());</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;">assert_eq!</span><span style="color: #7C7F93;">(</span><span>string</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">capacity</span><span style="color: #7C7F93;">(),</span><span style="color: #FE640B;"> 4</span><span style="color: #7C7F93;">);</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>string</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">shrink_to</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">2</span><span style="color: #7C7F93;">);</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;">assert!</span><span style="color: #7C7F93;">(</span><span>string</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">capacity</span><span style="color: #7C7F93;">()</span><span style="color: #179299;"> &gt;=</span><span style="color: #FE640B;"> 2</span><span style="color: #7C7F93;">);</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>string</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">shrink_to</span><span style="color: #7C7F93;">(</span><span style="color: #FE640B;">0</span><span style="color: #7C7F93;">);</span></span> <span class="giallo-l"><span style="color: #1E66F5;font-style: italic;">assert!</span><span style="color: #7C7F93;">(</span><span>string</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">capacity</span><span style="color: #7C7F93;">()</span><span style="color: #179299;"> &gt;=</span><span style="color: #FE640B;"> 1</span><span style="color: #7C7F93;">);</span></span></code></pre> <p>(<a rel="external" href="https://play.rust-lang.org/?version=stable&amp;mode=debug&amp;edition=2018&amp;gist=90112a3377f9d4a92f1c031f2b3bfc83">Rust Playground Link</a>)</p> <p>The code I worked on exposed the same methods on Artichoke's String classes: <a rel="external" href="https://github.com/artichoke/artichoke/pull/1505/files">Implements `shrink_to` on `spinoso_string::String` by briankung · Pull Request #1505 · artichoke/artichoke</a>. The actual code, excluding the doctests, was very minimal:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="rust"><span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;">#</span><span style="color: #7C7F93;">[</span><span style="color: #DF8E1D;font-style: italic;">inline</span><span style="color: #7C7F93;">]</span></span> <span class="giallo-l"><span style="color: #8839EF;">pub fn</span><span style="color: #1E66F5;font-style: italic;"> shrink_to</span><span style="color: #7C7F93;">(</span><span style="color: #179299;">&amp;</span><span style="color: #8839EF;">mut</span><span style="color: #D20F39;"> self</span><span style="color: #7C7F93;">,</span><span style="color: #E64553;"> min_capacity</span><span style="color: #179299;">:</span><span style="color: #8839EF;"> usize</span><span style="color: #7C7F93;">) {</span></span> <span class="giallo-l"><span style="color: #D20F39;"> self</span><span style="color: #179299;">.</span><span>buf</span><span style="color: #179299;">.</span><span style="color: #1E66F5;font-style: italic;">shrink_to</span><span style="color: #7C7F93;">(</span><span>min_capacity</span><span style="color: #7C7F93;">);</span></span> <span class="giallo-l"><span style="color: #7C7F93;">}</span></span></code></pre><figure> <p>[](/assets/images/screen-shot-2021-11-20-at-4.46.04-pm.png) <figcaption> <p>After a few tweaks, my PR was good to go!</p> </figcaption> </figure> <p>But what I liked about this was how <em>easy</em> @lopopolo made it to contribute! From giving me the initial nudge and motivation to build Artichoke locally for the first time to the extensive guidance in the issue, it was a great way to get some open source experience on a non-trivial Rust project. I can't recommend it enough. In fact, I can point you to a few issues that could use a similar treatment:</p> <figure> <p><a rel="external" href="https://github.com/artichoke/artichoke/issues?q=is%3Aissue+is%3Aopen+shrink_to"><img src="/assets/images/screen-shot-2021-11-20-at-22.20.48.png" alt="Image of similar issues on artichoke ruby" /></a></p> <figcaption> <p>These would require basically the same code changes as the above, and they are still open as of Saturday, November 20th, 2021!</p> </figcaption> </figure> <p>This was a great experience for me, and @lopopolo's stewardship of the project reminded me of both Ruby's value of niceness and Rust's value of inclusivity. It led directly to a <a rel="external" href="https://github.com/artichoke/rand_mt/pull/116">second PR</a> and I hope to keep contributing in the future to learn more about both Rust and Ruby.</p> <p>Links:</p> <ul> <li>The Artichoke Ruby project: <a rel="external" href="https://github.com/artichoke/artichoke">https://github.com/artichoke/artichoke</a></li> <li>The Artichoke public Discord server: <a rel="external" href="https://discord.gg/QCe2tp2">https://discord.gg/QCe2tp2</a></li> </ul> 2021-11-16T20:53:55+00:00 2021-11-16T20:53:55+00:00 Unknown https://briankung.dev/2021/11/16/pipes-in-elixir/ <p>Pipes are a delightful feature in Elixir that you can use to pass the evaluation of an expression into a function. For example, the code below will output "Hello, world!" to the terminal:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="elixir"><span class="giallo-l"><span style="color: #40A02B;">&quot;Hello, world!&quot;</span></span> <span class="giallo-l"><span style="color: #179299;">|&gt;</span><span style="color: #DF8E1D;font-style: italic;"> IO</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">inspect</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"># &quot;Hello, world!&quot;</span></span></code></pre> <p>You can use this to chain together different operations:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="elixir"><span class="giallo-l"><span style="color: #40A02B;">&quot;Hello, world!&quot;</span></span> <span class="giallo-l"><span style="color: #179299;">|&gt;</span><span style="color: #DF8E1D;font-style: italic;"> String</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">reverse</span></span> <span class="giallo-l"><span style="color: #179299;">|&gt;</span><span style="color: #DF8E1D;font-style: italic;"> IO</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">inspect</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"># &quot;!dlrow ,olleH&quot;</span></span></code></pre> <p>And Elixir specifically substitutes the result of the expression into the <em>first</em> argument of the following function, so you can still tack arguments onto the end:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="elixir"><span class="giallo-l"><span style="color: #40A02B;">&quot;Hello, world!&quot;</span></span> <span class="giallo-l"><span style="color: #179299;">|&gt;</span><span style="color: #DF8E1D;font-style: italic;"> String</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">reverse</span></span> <span class="giallo-l"><span style="color: #179299;">|&gt;</span><span style="color: #DF8E1D;font-style: italic;"> String</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">upcase</span></span> <span class="giallo-l"><span style="color: #179299;">|&gt;</span><span style="color: #DF8E1D;font-style: italic;"> IO</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">inspect</span><span style="color: #7C7F93;">(</span><span>label</span><span style="color: #7C7F93;">:</span><span style="color: #40A02B;"> &quot;SHOUTING BACKWARDS&quot;</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;font-style: italic;"># SHOUTING BACKWARDS: &quot;!DLROW ,OLLEH&quot;</span></span></code></pre> <p>Pipes can help visually represent the path data takes as it flows through your application. At this point, I'm probably using them a bit too much, but it's just too much fun.</p> <p>This is cool because in a lot of languages, you can't represent how your data is processed in such an orderly fashion because you need to adhere to the order that the programming language understands it - inside out. For example, without pipes, the last example would look like this:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="elixir"><span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;">IO</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">inspect</span><span style="color: #7C7F93;">(</span><span style="color: #DF8E1D;font-style: italic;">String</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">upcase</span><span style="color: #7C7F93;">(</span><span style="color: #DF8E1D;font-style: italic;">String</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">reverse</span><span style="color: #7C7F93;">(</span><span style="color: #40A02B;">&quot;Hello, world!&quot;</span><span style="color: #7C7F93;">)),</span><span> label</span><span style="color: #7C7F93;">:</span><span style="color: #40A02B;"> &quot;SHOUTING BACKWARDS&quot;</span><span style="color: #7C7F93;">)</span></span></code></pre> <p>Or, with some indentation to make it somewhat easier on the eyes:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="elixir"><span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;">IO</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">inspect</span><span style="color: #7C7F93;">(</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> String</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">upcase</span><span style="color: #7C7F93;">(</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> String</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">reverse</span><span style="color: #7C7F93;">(</span><span style="color: #40A02B;">&quot;Hello, world!&quot;</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #7C7F93;"> ),</span></span> <span class="giallo-l"><span> label</span><span style="color: #7C7F93;">:</span><span style="color: #40A02B;"> &quot;SHOUTING BACKWARDS&quot;</span></span> <span class="giallo-l"><span style="color: #7C7F93;">)</span></span></code></pre> <p>Typically, a programming language has <em>functions</em> that operate on certain <em>inputs</em>, but, like mathematical functions, those inputs are entered inside the parentheses. If you remember the form <code>f(x) = x + 1</code> from math class, then you might understand that to solve for <code>g(f(x))</code>, you need to read it backwards in order to evaluate it: first, you must know what <code>x</code> is, then you have to solve <code>f(x)</code> in order to finally solve for <code>g(x)</code>. The way programming languages operate is similar, except that it's the computer that's doing the solving, or evaluation, of your code. Pipes let you invert that and visually represent it as "first x, then apply f, then g" or <code>x ➡️ f `➡️` g</code>, to use pseudo-code.</p> <p>Feel free to play with these examples here: <a rel="external" href="https://replit.com/@briankung/Pipes-in-Elixir">https://replit.com/@briankung/Pipes-in-Elixir</a></p> 2021-11-01T22:55:03+00:00 2021-11-01T22:55:03+00:00 Unknown https://briankung.dev/2021/11/01/why-elixir/ <p><a rel="external" href="https://elixir-lang.org/">Elixir</a>, other than being an alchemical cure-all, does double duty as a programming language that's good for fast, resilient, parallel computing. It has all the syntactical charm of Ruby, with the power of an alien (actually Swedish, but close enough) technology. But why am I learning Elixir now, after so many years of Ruby?</p> <p>To understand why Elixir is particularly compelling to learn right now, you have to know a bit of the context of computing. For a while, from roughly the 1960s through the early 2000s, there was a golden period of computing hardware advancements in which processing "power" doubled every two years. It was fantastic unless you liked keeping your computer for longer than a year. However, unlike power levels in Dragonball Z, processing power is nuanced.</p> <p>What everyone thought of as processing "power" was actually the clock rate, or how many operations a processor could do in a second, and the doubling technically referred to the number of transistors in a processor doubling every 18 months, commonly known as <a rel="external" href="https://en.wikipedia.org/wiki/Moore%27s_law">Moore's Law</a>. But processors weren't being used the way we use computers nowadays - with hundreds, or thousands of things going on at the same time. The way some people use a single browser window with hundreds of tabs would be impossible on the hardware of yesterday. But that's rarely the only thing you have going on your computer at once - and that's what you know of! Our computers are always hard at work under our very noses, and the hardware has to support all that multitasking.</p> <p>Your computer is like a busy restaurant kitchen. Whatever the customer orders has to show up on time, but there are a thousand other things that have to happen meanwhile. Vegetables need to be chopped, soup stock heated, meat sliced. When things go wrong, a backlog of orders threatens to swamp the kitchen, slowing everything down. There are a few different ways a computer - or a kitchen - can handle workloads.</p> <p><strong>Sequential Computing</strong></p> <p>The kitchen has one cook. Let's say they can cook 1 meal per minute. That's a fast cook! But since there's only one cook, they can only work on one thing at a time. So if you have 60 people waiting on their meals, the last person is going to get their meal in an hour. That's no good. They might be a VIP or a restaurant critic.</p> <p><strong>Concurrent Computing</strong></p> <p>The kitchen still has one cook, but the cook is crafty and can juggle multiple things at once. Someone ordered a soup and a taco? No problem, they'll start heating the water for the soup, then they'll switch over to make the taco. By the time the taco is done, the water is hot and ready to go, so customers can get 1 taco <em>and</em> 1 soup in the time it takes to make just the soup.</p> <p><strong>Parallel Computing</strong></p> <p>Business is booming and even the blindingly fast, crafty cook can't keep up, so they hire more cooks - let's say 2 (for now). Now they can do twice the amount of work at the same time! And best of all, they can still use concurrent techniques to make efficient use of their time on multiple tasks.</p> <p>So what people thought of as "performance" doubling was actually just the single cook getting faster and faster. Which is great, but what if you want to watch YouTube while printing something and messaging your buddies? Well, that single cook - in our case, a single processor - is going to get overwhelmed fairly quickly.</p> <p>So processor manufacturers started adding cooks.</p> <p>You may have heard of processor cores. They are analogous to the cooks in our metaphor. Whereas processors of yore only had one core, processors now have sometimes over a hundred cores in the same piece of silicon. This abundance of "cooks" lets your processor speed things up in a different way than just running a single task faster. And what's more, processor manufacturers effectively <em>stopped being able to manufacture</em> cores that were faster on a pure operations-per-second basis, so they had no other choice if they were to remain competitive - they had hit the limitations of physics in terms of clock rates.</p> <p>Now, on the software side, many programming languages had only been designed to take advantage of one processor core at a time - sequential computing, basically. While processor cores were doubling in clock speed every 2 years, this wasn't much of a problem. If there was a performance problem, programmers could punt it down the line to their future counterparts who would have exponentially faster hardware at their disposal.</p> <p>But when the last gigahertz marketing campaign died, sequential computing wasn't too far behind. There are a lot of clever ways to get around this - <a rel="external" href="https://en.wikipedia.org/wiki/Asynchrony_(computer_programming)">asynchrony</a> and <a rel="external" href="https://en.wikipedia.org/wiki/Thread_(computing)">threads</a>, for example - but it turns out that it is not really something you can tack onto a programming language later. To go back to our analogy, if you have two chefs and they both try to use the ladle at the same time, a fight could break out, or if they both receive an order at same time, one might use up an ingredient before the other gets to it. A properly parallel programming language has to be able to orchestrate all of that at the same time.</p> <p>Enter <a rel="external" href="https://www.erlang.org/">Erlang</a>. Erlang, Elixir's parent language, so to speak, was invented from the get-go in 1986 to handle many thousands of concurrent connections because it was born in an environment that required it - telephony, the original social network. To crib a bit from <a rel="external" href="https://en.wikipedia.org/wiki/Erlang_(programming_language)">Wikipedia</a>, it specifically needed to be:</p> <ul> <li><a rel="external" href="https://en.wikipedia.org/wiki/Distributed_computing">Distributed</a></li> <li><a rel="external" href="https://en.wikipedia.org/wiki/Fault_tolerance">Fault-tolerant</a></li> <li><a rel="external" href="https://en.wikipedia.org/wiki/Soft_real-time">Soft real-time</a></li> <li><a rel="external" href="https://en.wikipedia.org/wiki/High_availability">Highly available</a>, <a rel="external" href="https://en.wikipedia.org/wiki/Uptime">non-stop</a> applications</li> <li><a rel="external" href="https://en.wikipedia.org/wiki/Hot_swapping#Software">Hot swapping</a>, where code can be changed without stopping a system.</li> </ul> <p>...in order to comfortably handle and maintain switches between phone users. One particularly important idea in Erlang is that <em>everything is a process</em>. You could <em>almost</em> say that <em>everything gets a single cook</em>.</p> <p>Elixir inherited all of that from Erlang, and a heavy dose of Ruby sensibilities from its creator, José Valim. Ruby and Elixir are both elegant programming languages for different reasons, but Elixir's inheritance means that it's better suited for parallel computing workloads that are becoming more and more important as hardware diversifies past the gigahertz wars.</p> <p>Elixir provides particularly compelling bait for web developers who are looking to provide near real-time user experiences while scaling up to possibly millions of users at the same time.</p> <p>If you're interested in learning Elixir, you can start at <a rel="external" href="https://elixir-lang.org/">https://elixir-lang.org/</a>. I can personally recommend Dave Thomas's book, <em><a rel="external" href="https://pragprog.com/titles/elixir16/programming-elixir-1-6/">Programming Elixir 1.6</a></em> for its wonderful exercises and clear, but concise writing style (if you're in the business of buying programming books, you can also save yourself a lot of time and money by getting your books through an ACM membership). You can also find Elixir on <a rel="external" href="https://elixir-slackin.herokuapp.com/">Slack</a> and <a rel="external" href="https://discord.gg/elixir">Discord</a>. Otherwise, I'm just starting on my Elixir journey, so feel free to geek out about it with me <a rel="external" href="https://www.twitter.com/briankung">@briankung</a> on Twitter!</p> 2021-10-31T21:43:38+00:00 2021-10-31T21:43:38+00:00 Unknown https://briankung.dev/2021/10/31/trying-elixir-as-a-rubyist/ <figure> <p><a href="/assets/images/trip-hazard-g42f62b8c1_640-e1635785693452.png"><img src="/assets/images/trip-hazard-g42f62b8c1_640-e1635785693452.png" alt="Warning sign of stick figure tripping over something" /></a></p> <figcaption> <p>Elixir is similar enough to Ruby to be familiar, but different enough to get tripped up by! <em>Image by <a rel="external" href="https://pixabay.com/users/openicons-28911/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=98658">OpenIcons</a> from <a rel="external" href="https://pixabay.com/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=98658">Pixabay</a></em></p> </figcaption> </figure> <p>I've been working through Dave Thomas's fantastic book <a rel="external" href="https://pragprog.com/titles/elixir16/programming-elixir-1-6/">Programming Elixir 1.6</a> in hopes of using Phoenix LiveView, and it has been really interesting, coming from Ruby. A lot of syntax is familiar enough to be alluring, but unfamiliar enough to trip me up. For instance, even something as simple as an <code>if</code> block was hard to write fluently:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="elixir"><span class="giallo-l"><span style="color: #8839EF;">if</span><span style="color: #D20F39;"> true</span><span style="color: #8839EF;"> do</span></span> <span class="giallo-l"><span> something</span></span> <span class="giallo-l"><span style="color: #8839EF;">else</span></span> <span class="giallo-l"><span> something_else</span></span> <span class="giallo-l"><span style="color: #8839EF;">end</span></span></code></pre> <p>Rubyists, did you catch it? <code>if</code> statements in Elixir have a <em>mandatory</em> <code>do...end</code> block. I got stuck for longer than I’d like to admit trying to compile my Elixir code only to have it complain about superfluous <code>end</code> keywords.</p> <p>As it turns out, this makes sense because <code>do...end</code> blocks are syntactic sugar for appending a keyword list to a function, and <code>if</code> is just a function in Elixir. But in Ruby, <code>do...end</code> blocks in <code>if</code> statements are not even valid (that I know of).</p> <p>I encountered a similar syntactical hiccup when converting short form function definitions to long ones in Elixir. Short function definitions follow this syntax:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="elixir"><span class="giallo-l"><span style="color: #8839EF;">def</span><span style="color: #1E66F5;font-style: italic;"> somefunc</span><span style="color: #7C7F93;">(</span><span>arg</span><span style="color: #7C7F93;">),</span><span> do:</span><span style="color: #DF8E1D;font-style: italic;"> IO</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">inspect</span><span style="color: #7C7F93;">(</span><span>arg</span><span style="color: #7C7F93;">)</span></span></code></pre> <p>…while long function definitions look like this:</p> <pre class="giallo" style="color: #4C4F69; background-color: #EFF1F5;"><code data-lang="elixir"><span class="giallo-l"><span style="color: #8839EF;">def</span><span style="color: #1E66F5;font-style: italic;"> somefunc</span><span style="color: #7C7F93;">(</span><span>arg</span><span style="color: #7C7F93;">)</span><span style="color: #8839EF;"> do</span></span> <span class="giallo-l"><span style="color: #DF8E1D;font-style: italic;"> IO</span><span style="color: #7C7F93;">.</span><span style="color: #1E66F5;font-style: italic;">inspect</span><span style="color: #7C7F93;">(</span><span>arg</span><span style="color: #7C7F93;">)</span></span> <span class="giallo-l"><span style="color: #8839EF;">end</span></span></code></pre> <p>Again, the differences are slight, but incompatible - and again, they make sense when you realize what the <code>do...end</code> syntactic sugar is for. The short form of function definition requires a comma after the function arguments, and the long form will throw a syntax error if you forget to delete the comma and simply move the function body into a <code>do...end</code> block. This is, as before, because the <code>do...end</code> block is simply sugar for <code>do: (function_body)</code> keyword list. But in order for it to be parsed as a keyword list in the short function definition, it needs the comma.</p> <p>Once again, it makes sense! But it’s a tripping hazard for Rubyists.</p> <p>Ironically, I couldn’t remember if the <code>do:</code> keyword list in the short function definition required the <code>end</code> keyword - even when writing this post! Spoiler: it is a syntax error.</p> <p>It’s also a bit of a paradigm shift to write in a fully functional language after writing object oriented, message-passing Ruby code for so long, but I’ll leave that for another post.</p> 2021-10-31T12:07:06+00:00 2021-10-31T12:07:06+00:00 Unknown https://briankung.dev/2021/10/31/hello-world-2/ <p>My homepage looks so lonely without a post that I thought I'd write something really quickly. It's been a long time since I wrote in a blog.</p> <p>I hope to jot down thoughts about a few of the things I've been working on lately here, including, but not limited to:</p> <ul> <li>Rust</li> <li>Elixir / Phoenix</li> <li>OS development (<a rel="external" href="https://os.phil-opp.com/">https://os.phil-opp.com/</a>)</li> <li>Ruby as I discover new, weird corners of it</li> </ul> <p>No doubt some hints of my humanity will leak out as well - the tagline is ⅔ non-programming, after all.</p>