Node.js Series: The Logging thing…


[Versão em Português]

This is part 3 of a series of posts dedicated on documenting my experiences adopting Node.js in the company so people out there can hopefully learn from my mistakes and have a good head start on their own efforts. If you would like to see the previous posts, please visit them here, enjoy and do let me know what you think!

Logging is extremely important, that you already knew. I have spent the last 10 years of my career dealing with integration tools that were not exactly high performance, were not exactly forgiving to your money on infrastructure and specially on licensing, but man, can they log. So much most database teams dealing with such products would have weekly if not daily base cleanup batches so they could keep up with the amount of information, most of it actually not that necessary, those products create.

Dealing with raw coding demands your team is always aware of how important logging the steps of their processes is and given the team was, just like me, coming from the private product word, it took us a little while realizing exactly how much logging was the very basic and how much was too much. You will find several articles about that on the web, but in the end your business will most of the time dictate what you could be logging, what you should and what you most definitely MUST log. Our business actually demands quite a bit of heavy logging, and they must be kept for a few years, so we needed a simple, but very high capacity tool to accept our logs and store them safely.

That is exactly when winston logging came to our playground, like they say it themselves, a logger for just about everything. Winston IS a very reliable logging tool but there are a few things you should consider before npm installing and forcing it into your production environment, and those are the options I want to tell you about.

Right off the bat you will see winston presents to you very nice and simple examples on logging to files, and even though this may attend most, when you are dealing with the high concurrency micro-services will face on enterprise solutions, the file logging will probably not be the best choice for you.

Throughout my career I have seen many companies rely solely on file logging solution leading them to see their systems not only being overwhelmed by high throughput but also being forced into investing on amazingly high cost storage devices because they failed either to foresee this or simply reviewing their strategy.

Cloud providers are specially unforgiving with this kind of strategy, if you are lazy at defining how you use your resources, expect BIG invoices by the end of the month.

Winston however is prepared that most people will not find it an enterprise ready solution, so they developed the concept of Transports. So when you require your winston into your code like that:

const winston = require("winston");

You will also have to sign one or more transports, like the file logger on this example:

const logger = winston.createLogger({
  level: 'debug',
  format: winston.format.json(),
  transports: [
    new winston.transports.File({ filename: 'output.log' })

And that is the beauty of it, at this point you can choose your favorite tool that will receive your data. And they have a WIDE variety of Transports available for your choosing here. It was however my case that my solution design would not fit any of those, not that they were not good, but as I said, this tool allows great flexibility and as I had a main solution blueprint beforehand that would demand a messaging server receiving the logs, I did not have to change it, I just had to customize my very own Transport, and it was for RabbitMQ.

This path was chosen for I intended a little data transformation on the logs before storage and I did not wish to see my services performance degraded by these, so my logs were directed to a RabbitMQ instance. If you think your solution may need this, you can download the transport right here, in case you need a little customization, feel free to fork it here.

In case you were wondering my logs last resting place is a MongoDB instance, where it is quite easy to search, you, however might need a different solution! Do let me know how you solved it though!

That was another episode on our migration to Node.js crusade, to know more and to stay up-to-date on this series, please follow me on twitter @vrsbrazil.

Thank you!


One thought on “Node.js Series: The Logging thing…”

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s