Scodec for Scala 3
Please log in to watch this conference skillscast.
Scala 3 introduces new features which help manage complexity. In this talk, we’ll look at porting Scodec from Scala 2 to Scala 3, using new language features to simplify the library.
You’ll see the ease of migrating projects to Scala 3 and perhaps be inspired to port some of your own.
Q&A
Question: The casts in your codecs - do you think that scala 3 will improve to the point where they are no longer necessary?
Answer: I don’t think so — there are a few in the scodec code base that should go away as Scala 3 improves, but the ones I showed in the slides I think will be necessary, as the compiler isn’t aware of the type equalities we claim at type level and how it impacts term level. As I understand it, these types of issues are common in more formal systems (e.g. Coq/Agda) but don’t quote me on it! :)
Question: Not directly related to Scala 3, but how much of scodec is about the binary representation versus the recursive-isomorphism-oriented generic code? I'm wondering how much can it share with JSON, CSV, etc. codec libraries.
Answer: Mostly about binary structure, especially given Scala 3's meta-programming capabilities. Before Scala 3, we definitely needed things like Shapeless and Magnolia to help make meta-programming possible, and some of the type level operators in scodec could have been ported directly to Shapeless (as an example). Another interesting thing to study is the variance model -- e.g. Encoder is contravariant, Decoder is covariant, Codec is invariant. So what does it mean to try to map a Codec? Since it's invariant, that's really like imap/xmap from InvariantFunctor. Now ask a similar question for flatMap - this leads to a blog series I wrote up, which eventually led to the adoption of invariant functors in scalaz 7 and later the adoption of more invariant structures in Cats. https://mpilquist.github.io/blog/2015/06/18/invariant-shadows/
Question: Thank you! I've tried writing an isomorphism oriented (as in impossible to write an Encoder without a matching Decoder) JSON library but got stuck in Shapeless quite soon, now I might give it another try with Scala 3.
Answer: Oh that’s an interesting point. Iso could definitely be made a stand-alone microlibrary.
Question: Did you define your own Iso? Now I recall I brought Iso and Prism from monocle
Answer: Yep! https://github.com/scodec/scodec/blob/main/shared...
Question: Did you come across anything macro-related that you couldn’t replicate with Scala 3
Answer: Not in scodec, but I had a popular library called Simulacrum that's not possible to port to Scala 3 - https://github.com/typelevel/simulacrum
Question: Oh dear. So will you drop support for the library? I would have presumed (incorrectly) that the new Scala3 features would be very aligned with what Simulcrum supports?
Answer: No, you're right in your assumption. Simulacrum isn't needed in Scala 3 b/c equivalent functionality (better even) is basically built-in to the language now. Also, unrelated to that, there's a Scalafix for converting a Scala 2 code base that was using Simulacrum to a Scala 2/3 cross compiling code base, which basically just expands @typeclass
Starting dotty REPL...
scala> trait Semigroup[A] {
| def (x: A) |+| (y: A): A
| }
// defined trait Semigroup
scala> given Semigroup[Int] {
| def (x: Int) |+| (y: Int) = x + y
| }
// defined object givenSemigroupInt
scala> 1 |+| 2
val res0: Int = 3
Question: In your example, it seemed that the type-level code and term-level code were isomorphic. Are there situations (maybe more complicated situations) where they might differ?
Answer: One way I like to think of that symmetry you noticed is that the term level acts as a proof of the theorem stated by the type level.
Question: My question is less about the library per se but more around for you as a library author/maintainer how much you think Scala 3 has helped you in let’s say amount of work and ergonomics (for example reflected in lines of code/boilerplate reduction and enjoyment).
Answer: I don't expect to see the degree of effectiveness in general as I saw with scodec. The built-in support for arity-polymorphic tuples was a huge simplifier for scodec.
One bummer about Scala 3 as a library author is that we can't take advantage of Scala 3 only features for quite a while. On the flip side, I've found bugs in Scala 2 code just by compiling it with the Scala 3 compiler. And the error messages, especially about missing implicits, are sooo much better.
Also, I'm glad you mentioned enjoyment. Working in Scala 3 reminds me a lot of my early Scala days, trying out new features and new ideas, seeing what works and what doesn't, finding new ways to express designs, etc.
YOU MAY ALSO LIKE:
Scodec for Scala 3
Michael Pilquist
Distinguished EngineerComcast