In The Belly Of The Whale: Tales From Haskell In The Enterprise
Please log in to watch this conference skillscast.
What is it like to bring Haskell to a large enterprise with an established culture? There are success stories for specialized teams, but what about the mainstream? We've been working in the trenches alongside enterprise developers doing Haskell for nearly 3 years and have lots to report. Ranging from the Haskell language and the ecosystem around it to how it has impacted the systems and people working with it, we will examine the journey from an introduction that took us into stormy seas and the vision of clear skies ahead.
Q&A
Question: I'm lucky enough to work for a business that "gets it", but the two main concerns I hear from not-yet-FP business are:
- How do you convince the business to run that first experiment?
- You mentioned that recruitment was hard for enterprise haskellers. The anecdotes I've heard is that you tend to get FPers beating down your door if you post a job ad. But it sounds like that's not your experience: is it hiring enterprise hasekllers that's hard, or is it just haskellers? Is it hard for the "standard" recruitment agencies (used by enterprises) to fill these needs?
Answer: As far as convincing the business, we succeeded by first getting the leadership to trust us as software engineers first. Then when we came and recommended Haskell they were at least primed to listen.
We also pitched them a project that was not really an experiment, but would end with replacing legacy COBOL systems that the business had been trying to replace for years. So the project value was there.
Definitely cultivating strong, trusting relationships with key leaders was essential
Question: When writing libraries, what's your approach
regarding pushing abstractions on the library users? Eg. the library offers
something A that can be seen as a Monoid, do you just expose a
instance Monoid A to the library user or you expose your own
(monomorphic) set of functions operating on A ? what if it was an
Applicative ? or a Comonad ?
Answer: Great question. For commonly used things like
Monoid and Applicative we will just go ahead and
expose instances as long as that's the most ergonomic answer. For more exotic
things (e.g. Comonad or free appliactives) we will try to wrap them
up and hide them.
Question: Thanks David. Clearly there's a spectrum of abstractions going from well-known to obscure, and this depends heavily on the developer background.
Answer: That is a great point. And if we can expose instances
of things, that is generally more acceptable than creating abstractions that
require users to use something. Saying "here is an instance of
Profunctor for our X" is different than saying "callers need to
use/understand Profunctor", just as an example.
Question: My favourite example is a graph parametrised on the
vertex and edge types, Graph v e . Should I expose mapNodes
:: (a -> b) -> Graph a e -> Graph b e and mapEdges :: (a -> b) ->
Graph v a -> Graph v bor just say it's a bifunctor? anyone not familiar
with bifunctors would balk at the abstraction feeling it's unnecessary. Maybe we
can to both and also expose mapNodes and mapEdges in
terms of fst and snd? And of course
bifunctor is just an example, insert your favourite type class.
Answer: That's a great example. Having mapNodes
and mapEdges explicitly defined and present in the docs is a huge
help to newcomers. It's hard for them to hunt down how do so something if it's
only exposed as a typeclass instance they're not familiar with. That doesn't
stop you from also providing whatever typeclass instances are possible
though.
I usually try to just write the functions I want for data types, and then package them up as typeclasses instances afterwards with instances such as
instance Applicative Foo where (<*>) = applyFoo
Question: yes, perhaps providing both would be the best option
(and perhaps it would be more practical to do fst = mapNodes rather
than vice-versa). As @Don
Syme was mentioning yesterday, discoverability is an important part.
Answer: Yes -- discoverability is a great way to think about it.
Question: How much has availability of libraries been an issue with things you needed to do in the Enterprise?
You can always write it yourself but often saying you need to write lib X before delivering feature Y is a hard sell.
Answer: It's definitely come up, but hasn't been a significant issue so far. There are libraries for most everyday tasks we want to do. Occasionally some of those libraries aren't ideal for newcomers though.
We've managed to walk a fine line with this. For a great number of things we are able to use libraries that already exist, but we have also written things to make our lives easier where applicable.
Question: I'm curious to know what patterns you use in your
enterprise codebases that you've found work well to keep Haskell approachable.
What effect patterns do you use? e.g. the ReaderT pattern,
mtl-style, etc. Do you use lenses for deeply nested data types?
Answer: So far we've found our lens code to be less maintainable than q the equivalent non lens-based code, so we actually try to avoid/remove lenses at this point.
Our monad stacks tend to be newtypes around ReaderT (and sometimes
a library's transformer when required), and then providing typeclass instances
for the monad type as required. So very much like MTL, but with a limited number
of transformer layers.
We definitely try to keep to a single form of error handling in any given monad.
Keeping smaller pure functions definitely helps. As is favoring more explicit style rather than having lots of typeclasses. Reducing things that are happening "magically" - like template haskell/quasiquoters
Question: I'd be interested to know why David and Trevis found hiring hard, then: was it finding enterprise-friendly Haskellers, finding people who would be willing to pick up Haskell along with their other duties, teaching their recruiters how to reach the Haskell communities, or something else entirely?
Answer: It's definitely helping the enterprise to learn how to find and hire the right Haskell developers. Enterprise hiring carries a lot more with it than a small business, and the enterprise might have to make compromises, like not requiring relocation as part of hiring.
Question: So it's a bit of "helping the enterprise become comfortable with things like relocation". That makes sense. And I suppose "rightness" means hiring people who aren't immediately going to apply the ideas from the latest conference preprint to a core business problem. What else do you look for?
Answer: Yeah, that's exactly right. We would really like to see the enterprise hire good software engineers who happen to know (or be interested in) Haskell, rather than someone who knows Haskell really well, but not how to apply it as part of a team.
Being good team members with empathy for those working with you and that not everyone is going to be interested in very advanced things is highly important.
Question: Do you find it easier to teach advanced Haskellers how to choose their tools carefully, or train people who "get" enterprise development into the ways of Haskell?
Answer: The real key is how much empathy the person has. If you're an advanced Haskeller but you can understand that most of those around you won't be then that's an easy sell. Similarly if you are an experienced enterprise developer but empathetic to the problems Haskell helps solve, that can also be an easy sell.
To be honest, we've had both successes and challenges with advanced Haskellers, so it really depends on judging the person. It can be hard to judge via interviews too, so that contributes to hiring challenges as well.
Question: Follow up from the library question: what big gaps in our library ecosystem have you encountered in the projects you've worked on?
Answer: The biggest gap we encounter is more around usability than functionality. For the most part we can usually find a library that does what we need, but whether or not that library is approachable to newcomers or not is another question.
Some of the biggest gaps are from having libraries that allow the users to stick to more "Simple Haskell" that remains accessible for those that are coming into the language
Question: How much do you think the "usability gap" can be solved via documentation / examples / tutorials?
Answer: Certainly better documentation will help, but if the
library requires you to turn on TypeOperators,
DataKinds and the like just to use it, we've observed that it
impacts usability in ways that go beyond examples and tutorials. Now error
messages are harder to follow and the code becomes harder to maintain.
Similarly, some libraries require you to use lens, which is
basically asking a newcomer to learn a new sub-language of Haskell
Also being very operator heavy without having a named function backing it can be quite harmful for newcomers.
I really like what Purescript did there in requiring a named function for the operator to be an alias.
YOU MAY ALSO LIKE:
Thank you to our sponsors and partners
Platinum
Silver
Bronze
In The Belly Of The Whale: Tales From Haskell In The Enterprise
Trevis Elser
Software Engineer
Flipstone Technology Partners, Inc.
David Vollbracht
PartnerFlipstone Technology Partners, Inc.