so-long-and-thanks-for-all-the-standard-fish.md

#607
Raw
Author
winny
Created
Dec. 9, 2022, 5:22 a.m.
Expires
Never
Size
11.5 KB
Hits
59
Syntax
markdown
Private
✓ Yes
You are viewing the source code of this paste. Click "Render" to see the formatted version.
# Table of Contents

1.  [So Long and thanks for all the standard-fish](#orgad89805)
    1.  [History](#orga6e1bd3)
    2.  [Production Racket: What would have to change?](#org89c4741)
        1.  [Cater towards CI/CD](#org42165d2)
        2.  [Protect users from themselves](#orgc6a07bd)
        3.  [How the heck do signature and units work](#org26890f2)
        4.  [Consider the role of documentation](#org6dedd2c)
        5.  [There needs to be transparency](#orgb5577d3)
    3.  [What's next?](#org0a2e8d4)


<a id="orgad89805"></a>

# TODO So Long and thanks for all the standard-fish

Hey everyone, I wanted to share with you a realization I had about my time with
Racket - it's time to move on.


<a id="orga6e1bd3"></a>

## History

Back in 2015-2016 when attending community college, I had discovered Scheme via
the R5RS paper, and was impressed with the conciseness of the language.  After
trying out a few implementations, it occurred to me the majority of scheme
implementations are not beginner friendly, missing documentation, and hard to
imagine how to use as a prospective user.  Then I discovered Racket.  It was
like some sort of drug - I couldn't get enough of it.  Racket does a lot of
things right.  Contracts or gradual typing means you usually have unambiguous
runtime errors without any typing problems.  Pattern matching and for loops
reduce code complexity significantly.  Most importantly, s-expression syntax is
dead simple and consistent - as long as you can google the first form in each
list (pair of parenthesis), you can understand a Racket program.

I toiled away for a couple years during university reading Racket
documentation, using it for IEEE-CS (Computer Science Club) HackerRank
competitions, even building a few tools I still use today.  Eventually I
started contributing a patch here and there to Racket - mainly to fix weird
stuff like build failures or documentation typos.  I also found opportunity to
write a riposte-like clone in Racket for my capstone project and began
uploading packages to pkgs.racket-lang.org.

Another couple years pass and finally I understand macros enough to use them on
my own.  These aren't easy to understand - the docs don't really help in this
regard.  But I persisted.  Soon after I started writing languages in racket
such as TinyBasic and subby (Subnautica item description language).

While this was all going on, I also started a project to build out tutorials or
howtos for Racket.  You see Racket suffers from a bunch of smart people who
themselves are deeply in the clutches of "The Curse of Knowledge".<sup><a id="fnr.1" class="footref" href="#fn.1" role="doc-backlink">1</a></sup>  They don't
quite understand what it takes for a beginner to pick up Racket, outside of a
pedagogical setting (classroom).  I even bought the `racket.wiki` domain with
the idea of calling it the Practical Racket Wiki - I wanted people to have
resources to do stuff quickly in racket without toiling through a metric
crapton of documentation pages.  Sometimes a productive dev needs to focus on
the important stuff, not how to use every single API in a language
correctly.<sup><a id="fnr.2" class="footref" href="#fn.2" role="doc-backlink">2</a></sup>

Then I had a realization.  It's been 7 years since I first started using Racket
and I'm still not being paid for research, writing software, nor making Racket
content.  I'm not getting rewarded for the time investment.  I need to be
realistic - Racket in production is a pipe dream.

This is me stating: I've had enough - I might use Racket for toy parsers or
other curiousities, but you'll never find me using Racket in a serious setting.
It's counterproductive and high risk.


<a id="org89c4741"></a>

## Production Racket: What would have to change?

Racket seems to work well for the research and pedagogical communities.  I
understand their needs may be fairly different than a production-seeking user.
The leadership is set up to cater towards a different community than the one
I was seeking to build.


<a id="org42165d2"></a>

### Cater towards CI/CD

Racket runs in docker, but there's more work to be done.  Racket does not have
a distinction between documentation dependencies and general dependencies.
This frequently causes Docker builds to take 10-20 minutes and use 2-5 times
the disk space.  This is caused by packages bundling documentation.  Want to
document your package and don't know any better, bam, you've made a package
that will ruin some production users day.  And the Racket community describes
this as a "human problem".  The solution is to split out packages so
documentation and library code are separated.  Alternately one may use binary
builds (which are built for a few particular ISA/OS combinations and only for
the latest Racket release).  The usual root cause is a dependency pulling in
`racket-doc` or `racket-index` when using a minimal racket distribution.  I
remember running Racket on a Pentium 4, with the horror of watching `raco pkg
install --auto some-random-lib` causing the laptop to spin for 10 hours, only
to fail due to a build failure caused by Racket's poor support for Alpine
linux.

I even voiced this and was flatly told contributions welcome.  Thanks, I will
totally invest a bunch of time guessing how the racket package server works
because it has no development instructions nor is there any public
documentation on how to automate deploys of this server process (such as an
ansible playbook or whatever).  So I'd pretty much be at the mercy of asking
upstream a bunch of questions and they're too busy to care about these
"trivial" problems.


<a id="orgc6a07bd"></a>

### Protect users from themselves

Racket has a blanket policy of never break APIs.  This sounds super cool in
theory.  Unfortunately I've seen how bad this can be in legacy APIs.  I worked
on OpenEdge Advanced Business Language codebases for about 8 months at one
job.  Now that, is the epitomy of bad API design and legacy carry over.  I
see flashes of that in Racket.

Instead of unifying a way to access and manipulate container types, we have
APIs for structs, hashes, sets, lists, vectors.  There are libraries to do
this, but nobody pushes them into the core language.  This leads to a ton of
frustration for beginners.  Please note there are excellent ways to unify these
and you can see them on the racket documentation - but nobody is pushing this
from a top-down perspective - so everyone is taught the least practical way to
write racket code for containers.

Racket currently doesn't respect Unix permission semantics out of the box.
This came as a shock because it seemed natural and inherit if you make a
programming language for Unix, it works like it should.  This seems like a
small issue, but what else am I going to discover the more I think about file
APIs?

Recently I discovered Racket's web server has a way to lookup headers in a case
sensitive fashion and a case insensitive fashion.  The former (case sensitive)
is abjet to the RFC.  The Racket devs made this into a human problems by adding
a caveat to the documentation, leaving it at that.  It's a techincal problem,
just unify the APIs and protect the user from themself.

There's also stale APIs I've never seen used anywhere, such as Racket's
`generator` API.  In short it's a way to build iterable objects in Racket
without using the standard scheme `stream` API.  This is a distraction to
anyone reading the documentation, and may even mislead readers into believing
they should use it over stream or a Racket class.

There's countless other sharp edges in the APIs like this.  For the most part
it's not so bad, however, it shows pattern of complancency of technical debt
in API design.


<a id="org26890f2"></a>

### How the heck do signature and units work

It's been 7 years and while I have played around with Racket's signature/units
API which is used for dynamic code loading.  It's very confusing and hard to
reason about.  It isn't understandable from the documentation alone.  This
bites because you'll need this to implement robust code hotloading and plugin
support in production.

I can't imagine getting other devs to understand this at all.  Maybe complete
rewrite is warranted?


<a id="org6dedd2c"></a>

### Consider the role of documentation

Racket is renowned for its thorough technical documenation.  If you need to
look up an API, it'll have documentation in most cases.  I get confused because
there isn't really a guide or tutorial for racket.  There's a super short and
sweet one and there's a guide which reads more like golang's tour (which is
certaintly not a guide either).

Working off this framework for documentation, I wonder if maybe there's a
misunderstanding of what it means to write tutorials, howtos, or guides.  Using
that framework, I believe the vast majority of racket docs are truly reference
material, not actually tutorials, guides, or how-tos.

I'd focus here, if there's truly a desire for adoption.


<a id="orgb5577d3"></a>

### There needs to be transparency

For the most part, without exception, I don't understand the Racket decision
making process.  There's not many policy documents nor a RFC process for Racket
itself.  I don't feel like I have a vote in how Racket grows, maybe I would
have a say if I joined one of the aformentioned academic cliques.

I don't feel incentivized to contribute upstream for the most part - they
aren't particularly easy to work with, and sometimes not the most personable -
we all have our days.  In other FOSS projects (VLC, FreeBSD, Homebrew, nixpkgs)
I've been blessed with a sort of advocate who helps guide new contributors.
Perhaps Racket could use some developer advocates - but that runs the risk of
giving beginners a voice, which I don't think the Racket people want.  They are
used to the complancey of complete creative control.

The majority of official racket codebases don't even have a readme or guide on
how to contribute.  It's a hurdle to know how to get started, let alone put in
energy on a PR that might not be well recieved.


<a id="org0a2e8d4"></a>

## What's next?

I will mark most of the code I have in Racket as "no maintenance intended".
Here's a badge!

The opportunity cost in my Racket time investment has been significant.  I've
been operating off the presumption I could either create a SaaS product around
Racket or find work doing Racket for somebody else.  Neither has come to
fruition, so I'm shelving most of my Racket work starting now.  If you want to
take over maintainership and direction of any of my racket codebases, please
reach out, I would be more than willing to transfer ownership.

Now I am presently learning more about the Clojure ecosystem with the express
idea of shipping stuff in Clojure.  Sure it doesn't have excellent error
messages nor a very good contribution process, however, it does have a
community of folks who have battle-tested this language and runtime in
production.  To be fair I have a lot of time spread across many languages so
it's not too important to pick up Clojure, however, I think it'd be a fun
technology to use in employment.

P.S. I am kind of broke working freelance.  If you want some devops work done,
backend work done, or R/D I'm your guy.

If you're interested in a blog post detailing some of my concerns with Racket
in production and the attitude of the core team with this use case, please let
me know on Twitter, Mastodon, or email.  Otherwise I'll quietly show myself the
door and let the past be in the past.


# Footnotes

<sup><a id="fn.1" href="#fnr.1">1</a></sup> 

<sup><a id="fn.2" href="#fnr.2">2</a></sup> I also argue that if it is possible to misuse an API, it's shit and you
should fix it thanks.