Fairness seems, at least to me, to be a topic of interest again. It seems that for a long while that the rules to fairness were clearly set, and that they were well understood. Recently, however, I have become under the belief that the keeper of fairness (the operating system) has lost some of the control in some cases, and has forced the common application developer to again need to ponder the fairness of their applications.

The situation in which I believe application developers need to consider fairness is in multi-threaded multi-session services. A good example of this type of application is a web service that supports multi-threaded actions, and the fairness concerns are relatively new. It used to be that a multi-threaded action would go and spawn new threads as needed. This old approach would result in the number of threads in the service be:

This works OK for systems that are only "slightly parallel", but if each session was working on a few hundred objects, and there were tens of sessions, which seems likely to be common in lots of domains, then if each object were to spawn its own thread, you would end up with thousands of threads running in parallel, which seems too big to me.

To a certain extent, the advent of Thread Pools in Java (my current language of choice, but I'm told C# is roughly the same) addressed the problem of spawning too many threads by creating pools that pulled jobs put into a queue. However, there seems to be a limit in this approach. If I wish to limit the total number of threads,    I cannot have each session own its own thread pool, but instead I need to have the sessions share a thread pool.This approach works well for optimizing the number of threads related to the number of processors and memory, but has a nasty side effect of having large jobs completely consume the resources of the pool, and cause jobs to be processed in a sequential order. (Transalation: if my session consumes the system for 10 minutes, everything started after my job sits until my job completes. To me, this doesn't seem fair; it would be much better to have jobs time slice (and all the other fancy OS-level things).

It might not be possible support preemptive multi-tasking in thread pools, but adding in some means to identify the job (session?) for which a given Callable  is associated would allow for the queue to be dynamically re-ordered to allow a sort of parallel job processing to occur in a kind of cooperative multi-tasking way.

While I could probably write an implementation of ExecutorService that would accept an additional argument to submit() that identified the calling session and queued accordingly, I'd hope that such a thing would be available in the core language, but I don't think it'll get there any time soon.