Lesson 8: No solution can be generalized to be always true in all situations. Depends on the system design.

We had a recent situation where we were going through a case where when the code reads from a value stored within a ConcurrentDictionary, this value could be stale.

We had an interesting discussion why this is ok. And more interesting about why we have to make an update to this value to be thread safe.

(1) Why Stale Value is OK in some cases

This is ok because in some cases, the design of the system is like a traffic light - some vehicles may go even when the light is orange, some may flout the "rules" and still the system can function.

If writes are consistent, reads can be stale in some systems. The critical reason why reads could be stale is if the stale value has no permanent effect on the system, and the cost of keeping it not stale is too expensive.

If the stale value caused permanent changes then it is not ok (persistence, database transactions, etc).

If the stale value is a signal which is read often and the effects of it being stale are very temporary in nature, for performance reasons, it is ok to read the stale value even while updating it in the ConcurrentDictionary.

I see this as a case where computers and programmers do not do well in grey area scenarios. Not all answers can be 0 or 1. Sometimes, grey areas exist and provide optimal solutions to real world problems.

(2) Why the update itself should be thread safe

If this ConcurrentDictionary stores a number, and we want to update the number, we want to keep this operation thread safe because if it is not, then depending on the number of threads accessing the write at the same time, the value of the number could careen across wrong values, thereby leading to a permanent bad state.

This is similar to the traffic signal entering a permanent state of malfunction where only some lanes turn green all the time for no particular reason except software error irrespective of the traffic load at each side of the signal.

We want the number to be correct on a permanent basis, but allow the reads to be stale sometimes for performance reasons.

A great way to do this is to use the AddOrUpdate method which I have not verified myself. But it looks like the below may work fine:

concurrentDictionary.AddOrUpdate(key, value, (key, oldValue) => oldValue + 1);

So locking the update in this case is not really being inconsistent when following this philosophy.