Cultivating an Engineering Dialect
Please log in to watch this conference skillscast.
Haskell has seen success in commercial environments, with teams of professional engineers choosing it for its claims of reliability, a rapid development pace, and easier maintenance over the long term. On top of that, a large community of hobbyist tinkerers and academic researchers are always releasing new and exciting abstractions, libraries, and language extensions, each offering improved ways to structure, build, and test our programs.
Engineering teams have diverse knowledge and skill levels, and new team members need to come up to speed to work effectively. This poses us a challenge: which abstractions, libraries, and language extensions should we choose from the ever-growing pool? How should we determine what level of Haskell to adopt? Should we always embrace the cutting-edge to squeeze out every advantage, leaving new hires in the dust? Should we reject novelty and focus only on the “simple” or “boring” ways of doing things, even if doing so gives up some potential effectiveness?
This talk will bring clarity to these questions. Rather than prescribe a uniform solution, we offer you the tools of thought to make informed, intentional decisions, and cultivate an engineering dialect that works for you.
Q&A
Question: What’s the biggest unexpected thing or mindshift you’ve had going from research Haskell --> corporate Haskell?
Answer: I worked on a research project for a year at the end of
my time in university, on a code base I inherited from my supervisor. The code
in that code base was simpler than "Simple Haskell"! Haskell written like it was
Miranda, with no open source dependencies, error for error
handling, and this sort of thing. When I took over, I used the coolest stuff I
knew about and learnt some of these lessons. I wasn't thinking in these terms at
that stage though, so I ended up with a bit of a mess I had to get myself out
of, and learnt some of these lessons.
A benefit of corporate Haskell is that there tends to be more than one person at a time working on a given code base, and I think having a team (and more broadly a community) is critical.
Question: What about containment/partition? You're talking about weight/power ratio as if it were distributed over the codebase, but do you have advice about how to contain the risk so some of the high weight code can be contained?
Answer: I think this is possible to some degree, eg. by hiding things behind library boundaries, so all the GADTs live under the bonnet or something. I don't like this argument so much though, because I think everyone on the team needs to be able to work on all of the team's code, so eventually the person who doesn't know GADTs will have to open the bonnet before they can close their ticket. I also think, if taken to extremes, this risks creating a sort of caste system, where the elites who know the mystical secrets get to work on all of the libraries, and then the unenlightened write the business logic in the middle.
Question : I think a good example of this is the boundary between application and operations, where the speed at which people learn new tools can be different in each side.
Not because Iove Nix, but when I saw it I thought "one of these things is not like the others"
Your Makefiles can be hairy, and the intern still can type make
foo.
I also wonder whether "everyone on the team needs to be able to work
on all of the team's code" is one of those rules you have to know when and why
to break.
In the Python world sometimes you have to go for ctypes and write some C glue, that's part of the job. But it's not the same language.
Constraints may be different in Haskell codebases.
Also I guess the risk of what you call "the caste system" may also be necessary as specialisation once your team grows.
It's ok to have someone write ML algorithms, someone else deploy them to production pipelines, and someone else write business logic using the results.
In your case, wouldn't it be ok for someone to maintain a DSL and others to consume it, if the team has grown enough? The libraries used in each side of this bargain might be different, and the risk calculus have different parameters.
Answer: They're probably similar to constraints in other languages, for the most part. GHC language extensions are a great example where they can change everything from the surface language to the typechecker, and everyone likes different sets of them.
Question: Not arguing, rather wondering whether we're thinking about different constraints. Seems like C++, where you have to agree on "which C++" before writing code, or you end up with widely different languages.
Answer: My talk mostly focused on a single team. I imagine these ideas would need subtle modification to work for multiple interacting teams (eg a DSL producer and consumer as you say)
Nix is a really complicated example for me. I have very strong mixed feelings
about it. I find it to be the best-in-class solution, but I also have found it
to have a non-negotiable learning cliff. If my primary role was deploying code,
I suppose I would have the time to sit down and learn Nix and deeply as I've
learnt Haskell, but I find that the only way to learn it is to read all four
massive manuals (which have no definitive mutual ordering), plus blog posts,
plus read 500 files called default.nix on github whenever I have a
problem to solve, plus hang out on IRC. I've found that the error messages are
inscrutable - usually when I make a simple mistake in my file, I will get an
error "string is not a function" deep in the guts of some other file I've never
heard of. Still, now that our Nix is in place, I can update and add dependencies
and so on, and we have scripts to do a bunch of common tasks, and Isaac and Dave
know Nix very deeply, so I have a community to draw on.
We've begun trying to document our nix usage in our handbook. It's been harder than Haskell. We need to put more effort into that problem soon.
I don't like Nix, and I'm glad we use it.
Question: What's your strategy for "There's this wacky technique but the one person attached to the technique that knows it well enough to teach it has left the company"?
If you have encountered that situation.
Answer: Right, I'm hoping that they did something like in this talk, where they linked to their favourite blog posts or talks before they left, and helped others learn. If not, I suppose you'll have to live with the pain of learning that technique unassisted, or rewrite the relevant parts of the code. Neither is usually very appealing :(
YOU MAY ALSO LIKE:
Thank you to our sponsors and partners
Platinum
Silver
Bronze
Cultivating an Engineering Dialect
George Wilson
Functional Programming EngineerPaidRight