Lodash FP usage retrospective


My current project is completing its third year. From the start, we’ve been using aggressively the Lodash FP library through our whole JS & TS codebase, whether it’s on the Back-End or Front-End. I recently performed a small analysis of our usage of the library to spot some weird usages that have slipped through code reviews and make a small retrospective about how this tool and functional programming are used in a mature production app.

The results of the analysis were sometimes surprising as some of the sanctified FP tools show little usage on our side, while some lesser-known or more basic functions are widely popular. Let’s dig in after a small digression about the lib itself.

Lodash… FP?

Lodash (https://lodash.com/) is a widely used library in the JavaScript ecosystem. It provides invaluable algorithmic tools that can save developers lines of code, time and bugs. Its less known sibling is Lodash/FP. As per the documentation, this build is providing “immutable auto-curried iteratee-first data-last methods.”. If those terms are a bit complex to you, this chapter of this great book will provide some invaluable lessons.

This lib is not the only contender nor the most advanced in the FP world but our team chose it because it’s much easier to train new team members with it. Most JS software developers have some experience with Lodash or Underscore and very few are familiar with the concepts behind Ramda or Pointfree-fantasy.

So, what are the biggest contenders

The code analysis focused on the number of imports of each Lodash function our main Web App. This is less precise than counting the number of usages of each function but this still gives a good representation of our usage. We grouped some of the functions as they share a common role.

get and getOr

This may come at a surprise, but we use get & getOr a lot (close to 200 imports with usually a lot of usage per import). They are by far the most used Lodash functions in our codebase. These are nice getters functions that allow to define a path for an attribute in a simple or complex object and retrieve the value.

const data = {
  a: {
    b : 1
  },
  c: 2
};

// FP variant puts the data as last argument (and this is great)
const b = get('a.b', data);

console.log(b); // 1

The first reaction to all newcomers is a big “Meh”, but after a short time, team members usually adopt it massively. I have always been doubtful with “advanced” accessors until I came across Lodash’s (probably because most of the accessors I saw in the past were used to perform side effects). We don’t have a specific policy to access all attributes like that, but it makes a lot of sense when using the FP variant of Lodash and a point-free style. These two functions have two pros and one con:

  • Con: typing attribute path inside a string always raises a warning in my heart. The linter is usually powerless to help us against a typo although TypeScript can perform some nice type inference. In our team, most of the get & getOr usages can be found either in redux selectors (where we have 100% test coverage policy) or in React components. Our experience is that years of usage of these functions did not translate the risk highlighted above into bugs.

  • Pro: They provide safeguards against a null or undefined value in the middle of your chain. The Optional Chaining EcmaScript proposal is about to nullify this pro. This has been very valuable for us, especially in conjunction with GraphQL where graph queries can easily return nulls.

  • Pro: The FP variant of these functions shines. Using builtin currying & reverse order of arguments, we can build easy to write and use getters around our code. To that purpose, we only have to call the get function with only the path as argument. It will return a nice function that can take any kind of compatible data structure.

import { get } from 'lodash/fp';

const data = {
  a: {
    b : 1
  }
};

// Here we put the currying into practice to build a getter function.
const getB = get('a.b');

// The function only need the last argument to be executed. 
// This is why we like data-last functions.
console.log(getB(data)); // 1

The getters can easily be extracted and shared. Naming those functions is often very valuable to abstract deep attribute access in data structures (think getUserNameFromToken). Its curry feature also leads to building many unary functions (functions that take only one argument) that are fantastic for function composition. It then does not come as a surprise that flow, a function composition tool is the second most used Lodash function in our code base.

flow

Flow comes next in our list (80 imports). This is a typical FP tool used for function composition (aka function centipede).

There are several ways to perform function composition, they are illustrated below with different implementations of the same function:


import { compose, flow } from 'lodash/fp';

const foo = num => num + 1;
const bar = num => num * 4;
const baz = num => num - 1;

// all these functions are equivalent
const composeManually = data => foo(bar(baz(data)));
const composeWithLodashCompose = compose(foo, bar, baz);
const composeWithLodashFlow = flow(baz, bar, foo); // also pipe

compose is often the classic tool for people coming from an FP background as it reads in the same way as the manual composition, but flow reads sequentially left to right and is, therefore, the first choice of all other people. It also reads the same way as a promise chain. The team made an early decision in favor of flow.

These tools are the best friend of point-free functional programming adepts. They work with unaries (see where we’re going…) and enable to write very readable and pure functions:

import { capitalize, filter, flow, get, map } from 'lodash/fp';

const userToken = {
  user: {
    permissionList: [
      {
        name: 'serviceA',
        enabled: true
      },
      {
        name: 'serviceB',
        enabled: false
      },
      {
        name: 'serviceC',
        enabled: true
      }
    ]
  }
};

const getEnabledPermissionList = flow(
  get('user.permissionList'),
  filter({enabled: true}),
  map('name')
);

console.log(getEnabledPermissionList(userToken)); // ["serviceA", "serviceC"]

//You can also extract all parts of your composition
const getUserPermissionList = get('user.permissionList');
const keepEnabledPermissions = filter({enabled: true});
const mapNames = map('name');

const getEnabledPermissionList2 = flow(
  getUserPermissionList,
  keepEnabledPermissions,
  mapNames
);

// Flow composes also nicely into other flows
const getUpperCaseEnabledPermissionList = flow(
  getEnabledPermissionList,
  capitalize
);

console.log(getUpperCaseEnabledPermissionList(userToken)); 
// ["SERVICEA", "SERVICEC"]

If you compare to chained APIs, this is incredibly superior. Each piece is testable individually and you can build and name intermediate functions to represent business concepts. Sometimes we use such a business name to convey meaning to very simple operations. We have no general rule about when to use a writing style that shows the reader how an operation is performed (map('propertyA')) or one that shows its meaning and abstracts the implementation (const formatForUI = capitalize).

Although it’s not mandatory to use pure functions, they provide a lot of benefits. First, it’s more testable and reusable but it also enables things like memoization to boost performance.

In our codebase, most of our redux selectors and data structure manipulation are built using flow. Every time an operation is expensive, the resulting function is wrapped with caching (using Lodash’s memoize, redux’s reselect or react memoization tools).

negate & friends

negate is our fifth most imported Lodash function. This can look original for something that dumb.

import { flow, isEmpty, negate } from 'lodash/fp';

const isEven = num => num % 2 == 0;
const isOdd = negate(isEven);
const isOddOldWay = num => !isEven(num);

const hasAtLeastOne = negate(isEmpty);

const hasAtLeastOneTruePermission = flow(
  getEnabledPermissionList, // reuse from above :)
  hasAtLeastOne
);

This is my experience that it’s better to build opposite functions based on only one implementation. In imperative programming, a small ! is often used, but as we are manipulating functions, having a function that wraps another one and returns the opposite boolean is very useful. Classic point-free style bonus, it also reads very well and is hard to typo.

This function is accompanied by a lot of small utilities that perform also dumb things like eq, isNull, isNil, and others. The spirit is the same. On the same occasion, we stopped spending time on the best way to detect null from undefined or checking is a number is really a number. Time is better spent elsewhere, believe me…

map vs reduce vs forEach

48 map, 5 reduce are 5 forEach. Wow, I didn’t expect to have so few reduces and so many forEach. After close examination, all the forEach are justified. If you are not familiar with those, they are the bread and butter of every FP article out there. One often unquoted benefit is the reduction in bug density due to the avoidance of index manipulation. In the same spirit, the team favors functional tools to perform direct access to specific elements in an array (head, tail) or array destructuring. But let’s go back to our iterators.

map usage seems pretty standard to me. The idea of a type transformation (think projection) applied to a list can be applied everywhere. One might wonder why we do not use the native Array.prototype.map. Again we don’t have a specific rule about it, but Lodash’s map applies to object and map collections, can use the builtin get style iterator and benefit from the curry/data-last FP combo. Of course, it means a lot of unaries easy to name, reuse, test and compose.

reduce might an FP star, but in the end, Lodash’s utilities, probably often built on top of reduce solves most of our use cases. I would still recommend the function for studying purposes. I was expecting that some of the heavy FP recipes that we use might be one day refactored in a high-performance unreadable piece of code relying on reduce or older fast loop tools, but, after some iterations on performance analysis, none of these have been flagged for a rewrite. Speaking of performance, we have what I would consider a high number of memoize imports in the codebase, especially after having most of the expensive stuff in redux selectors already using memoization techniques from the fantastic reselect library.

I have a personal hatred for forEach. I have countless times seen people use in code interview as a poor’s man map or reduce. The indication that it returns undefined should hint that something is off. My understanding of the function is that it should be used only to manage side effects (and indeed, all of our cases fall into this category after close examination).

In case you are asking yourselves, there is no while, for or for of statements in our project.

cond

I already wrote about cond earlier. I love the function and one might wonder why we only have 10 imports. The number of if and ternaries is much much bigger. That can be explained easily by the fact that we have very few complex branching in our codebase and the vast majority of them are using cond. Redux’s selector still relies on nice old switch statements.

FP specifics (constant, identity, tap, stubTrue, etc.)

Finally, there is a list of contenders that can seem very strange for an imperative programmer. These are mostly simple functional wrappers that fit well the API of not only our tools but all the JS ecosystem and base language. curry should need no introduction at this stage (if so, you’ve missed a link to a nice article in the Lodash… FP section).

constant returns a function that returns the same value it was created with. Its main role can be found in our cond functions. It can easily be replaced by a small arrow function like () => 2 but it for me it reduces the cognitive load to have plain English instead of a function expression and helps when talking about code.

import { cond, constant, identity, stubTrue } from 'lodash/fp';

const takeCorrectBranch = _.cond([
    [isMainCase, transformDataAccordingly],
    [isSpecialCase, constant(MY_SPECIAL_VALUE)],
    [isError, handleError],
    [stubTrue, identity] //stubTrue being often renamed as `otherwise`
]);

The example above also features stubTrue and identity. identity is used in a variety of situations like with a filter, groupBy or sortBy. Again, these tools can be replaced by simples functions like () => true and val => val but for the same reasons, we prefer the English term.

Let’s close this section by speaking a bit about tap. It’s bit more complex than the others since an implementation would be interceptorFunction => input => { interceptorFunction(input); return input; }. As you can see, it returns a function, that will forward the input (like identity), but it will execute the interceptor function with the value before forwarding it. It is used to trigger side effects in compositions like flow or in promises chains. We often wrap side effects with tap even if they already return their input when we want to signal at that the original data is forwarded and/or that a side effect is taking place.

import { flow, tap } from 'lodash/fp';

const addDataToMap = flow(
  toGeoJson,
  filter(isUseful),
  tap(logIt),
  tap(displayOnMap)
);

Even though you have no idea how the toGeoJson, isUseful, logIt and displayOnMap work, it’s easy to get an understanding of what the addDataToMap function does and what its API is.

Conclusion

Our global Lodash usage reflects a lot about how our team thinks and solves technical problems. We use a functional programming style to favor meaning over absolute code performance (which is tackled by other means). Adopting the language (a lodashy one in our case) is a bit hard for newcomers coming from an imperative world, but once acquired, it provides great benefits for maintainability, analysis, and team communication.

Appendix: whole usage list

Here is the whole list of our Lodash function imports in one of our Front-End codebase. If you are interested in some that I didn’t cover, feel free to contact me.

namecount
get166
flow80
map48
getOr28
negate22
isEqual22
isNil20
isEmpty18
isNull17
curry17
filter15
find14
keyBy12
orderBy10
memoize10
cond10
tap9
includes9
stubTrue7
noop7
sortBy6
pick6
identity6
head6
assign6
values5
set5
reduce5
forEach5
remove4
range4
partial4
last4
every4
take3
partialRight3
has3
groupBy3
contains3
concat3
zip2
uniqueId2
unionWith2
trim2
toNumber2
size2
reverse2
placeholder2
once2
mapValues2
isUndefined2
intersectionBy2
initial2
eq2
defaultTo2
constant2
zipAll1
xor1
valuesIn1
uniqBy1
uniq1
thru1
takeRight1
tail1
startsWith1
spread1
some1
slice1
reject1
rangeStep1
pickBy1
mergeWith1
max1
keys1
isNumber1
isNaN1
isFinite1
indexOf1
floor1
flatten1
flatMap1
findIndex1
equals1
differenceWith1
debounce1
compact1
ceil1
assignAll1