Eric's Tech Blog

I mentioned briefly in my last post about how the book I have been using, Node.JS, MongoDB and Angular JS Web Development, has a lot of deprecated code in it once I hit chapter 13. I do not want to teach myself code that is going to end up being trash here as the technology world evolves so whenever I got the deprecation warning, I actually went ahead and refactored all the source code from the book into the latest version using the documentation from MongoDB’s API. This way, I actually am learning the latest and official way to do things, and creating great reference code for me in the future to use (and for anyone who actually reads these blog posts or browses my GitHub as well). Chapter 15 was definitely the worst of it. First of all, the book starts using a new collection called “word_stats” that has over 4000 words in it. However, the book neglects to tell you where to get the collection data from. It pissed me off originally, and I actually was about to just skip to chapter 16 until, lo and behold, I noticed a massive generate_data.js file in the chapter 16 source code folder. I opened it up in VS Code and discovered thats the file that generates the schema and all the data for the word_stats collection. I decided to go ahead and give it a run and now I could start messing around with the source code in chapter 15. I was happy I did not have to skip that chapter. But the good news only lasted a short while as I went to run the source code and was met with the infamous DeprecationWarning message yet again:

Now for the past DeprecationWarning messages, the fix was usually simple. Something like change update to updateMany, so no sweat there. But in this particular message it tells me, “We recommend rewriting using the aggregation framework.” I was set to refactor the group method into the aggregation framework. Now since I have never messed with MongoDB before this book, it was kind of like trying to read a foreign language for the first time. Some wods you realized were close to English and could easily understand but the rest made no sense whatsoever. I have spent the past few days reading StackOverflow, the MongoDB docs and the MongoDB Node.js docs trying to understand what the hell is going on. Here’s a little sample to compare the refactoring for one of the examples.

doc_group.js [Book Source Code with Group]
----------------------------
collection.group(['first','last'], 
              {first:'o',last:{$in:['a','e','i','o','u']}},
              {"count":0}, 
              function (obj, prev) { prev.count++; }, true,
              function(err, results){
        console.log("\n'O' words grouped by first and last" +
                    " letter that end with a vowel: ");
        console.log(results);
  });

doc_group_refactored.js [My Refactored Code with Aggregation Framework]
---------------------------
collection.aggregate([
        {$match: {first: 'o', last:{$in:['a','e','i','o','u']}}}, 
        {$sortByCount: "$last"}    
]).toArray(function(err, docs){
        console.log("\n'O' words grouped by first and last" +
                        " letter that end with a vowel: ");
        console.log(docs);
});

So, to me, the syntax is way different. [collection].group() is has the following parameters: keys, query, initial, reduce, finalize, command, options, callback. The [collection].aggregate() function just takes an array of the steps in the pipeline as the only parameter and returns an array instead of an object so I have to use toArray to get the information I need. So we can break things down, but the first thing you do in aggregation is find a match. In our case, we need to match where the first letter is ‘o’ and the last letter is a vowel (in the array of vowels). Fortunately, the document has a field path for $first and $last letter of each word so luckily we can just copy that directly over from the group’s query parameter. It looks like they set a placeholder 0 for the count and then use the reduce parameter function to increment for each word it finds that matches the first set of items in the query parameter. Then after the count is done, it returns the whole object. I think I actually like the aggregation framework better now that I have practiced using it, plus I found the $sortByCount operator in the aggregation framework and once I understood how to use it, it automatically did the counting for me and was easy. There is no need for a reduce or finalize function, just one line of code and I can do the counting with the new aggregation framework. The output, however, is not as detailed but I think works just fine:

doc_group.js [Book Source Code with Group]
----------------------------
'O' words grouped by first and last letter that end with a vowel: 
[ { first: 'o', last: 'e', count: 21 },
  { first: 'o', last: 'o', count: 1 },
  { first: 'o', last: 'a', count: 1 } ]

doc_group_refactored.js [My Refactored Code with Aggregation Framework]
---------------------------
'O' words grouped by first and last letter that end with a vowel: 
[ { _id: 'e', count: 21 },
  { _id: 'a', count: 1 },
  { _id: 'o', count: 1 } ]

You can see above that the original code prints out the first and last letters to the output. But using $sortByCount, the item we are grouping by becomes _id so in our case that is the last letter. I would love to figure out how to add the first letter in there too (even though it really doesn’t matter) but every time I tried, I got null as a response so I will just leave it alone.In the book, they actually go over the aggregate method over the next few pages (after the group method), but the callback function is no longer a parameter like the book states so I still needed to refactor that code as well.

You can view the full source code for chapter 15 and all of my refactoring in my GitHub repo. Hopefully my refactoring can help someone else who needs to migrate from the group to the aggregation framework in MongoDB with Node.js. I think this is long enough of a post for a single chapter of a book, I will be learning Mongoose for Node.js next which is another plugin to connect to MongoDB. I probably will skip writing a post about that chapter unless something interesting happens, hopefully soon I will be getting into AngularJS!

“If you aren’t going forwards, you’re going backwards” the old adage goes. I recently picked up this book called Node.JS, MongoDB and Angular JS Web Development. I normally use PHP MVC, SQL Server and HTML5 for coding my web applications in a personal or work environment but I figured it is best to pick up on some of the most widely used languages that other companies use as well to help round out my technology education. I am familiar with Node.js from trying to create my own artificially intelligent chatbot a few years back using Wit.AI. However MongoDB and AngularJS I have heard about, but never actually done anything with them.

Learning MongoDB has been interesting, it is nothing like SQL Server or MySQL which I have used many times before. I have successfully launched the MongoDB daemon, created a user and database admin and turned on authentication for the database.

The next section of this book starts discussing how to connect Node.js to MongoDB, however at the time the book was published, they must have used an older version of the MongoClient but never listed the version they use in this book. Therefore much of the code does not actually work and I was left to search online with the official MongoDB API Documentation to try and figure out how to make a connection and perform operations. I’m not entirely mad about this, I am still learning the ropes, but it definitely can be an obstacle to some who just buy the book hoping its a one stop shop and all of a sudden nothing is working for them (and many people actually say that in the reviews I have seen online for the book).

It definitely does present more of a challenge to not be able to word for word use the books source code but to look at the book and then compare to official documentation and refactor the code as needed to make it work. Once I begin to hit chapter 13 dealing with all the Node.JS and MongoClient code, a lot of the books code was deprecated so I needed to refactor a lot of it on my own for it to work. With VS Code, I can get tooltips that guide me with the official documentation, which is SUPER helpful:

Starting in Chapter 14, I learned how to do a whole bunch of things to documents in a MongoDB collection such as: creating, inserting, saving, updating.etc. The book makes a separate file for each of these methods, I decided to make one giant document.js and you can pass a parameter such as ‘save’ or ‘update’ and it uses the correct method to do so. Plus the code is refactored up to the current 3.2 version.

I have this chapter and all of the code for my practice in my GitHub at this repo. At this point, I have created one rather lengthy blog post since Chapter 14 was quite a long chapter so I will stop this post here and continue Chapter 15 in the next post!