Application logs are a rich source of data which can answer questions
about performance, usability, and failures of a production system.
However, logs are often generated and stored in an unstructured form
distributed across multiple machines, making it difficult to query the
data to answer those questions. It is better to stream logs from
multiple machines to a single central server which then presents the
data in a searchable form and provides graphing mechanisms for
numerical data within the logs. This gives a much faster, more
intuitive way of understanding what your logs are telling you.
This talk presents a lightweight reimplementation of the log streaming
tool logstash, based on ClojureScript, core.async and node.js. By
using node.js, it is able to stream from disk or network sources to
remote redis or elasticsearch instances. The combination of
ClojureScript and core.async enables a clearer abstraction of
communicating sequential processes (CSP) rather than raw node.js and
its stacked up callbacks. This provides advantages in expressivity,
flow control, and memory management.
When developing an application in Java, there are generally
three main options to track a bug:
- printing the state of various variables and analyse the logs
- use of debugger to carry out step by step analysis of the code behaviour.
- expose your objects, via a transport like JMX or a rest layer, to
read or interact with them
Some bugs only appear in production size environments. It can be slow
or unpractical to apply any of the above techniques to carry out an
investigation. Instead, would it not be convenient to just be able to
interact with the deployed JVMs individually and ask them questions or
trigger some actions without having premeditated them?
An alternative exists: a REPL (like Clojure Nrepl) provides an
interactive prompt onto the JVM. It can be used to launch all sorts of
commands to introspect inside the content of the process e.g. to work
out, at run time, what values are contained inside objects. It can
also be used to inject code, create new classes, instantiate objects,
call methods and traverse various chains/collections of objects to
carry out live debugging at run time. Even private members can be
accessed. Anything that could be coded in Java is accessible without re-deployment thanks to the dynamic nature of the Clojure language.