GTask: Add g_task_set_scheduling()
My application (OSTree) has suffered really bad performance
degredation recently, and doing some analysis of the problem, I think
a lot stems from the recent order of magnitude increase in the default
GLib worker threads.
OSTree does a *lot* of async I/O (directory enumeration, calling
link(), reading/writing files) etc. It very quickly reaches 100
threads, but there is up/down thread churn as threads complete, but
process gets I/O bound and thus the threads are starved of work, but
then the main thread manages to fill the queue more, suddenly spinning
up lots of worker threads again.
Now, as this patch forces users to specify, there are several distinct
classes of tasks. It basically never makes sense to run more
CPU-bound threads than there are CPUs. Spawning 90+ threads to do
SHA256 calculation for example is just creating pointless extra
contention on a dual-processor system.
Therefore, the limit of the thread pool for G_TASK_THREAD_KIND_CPU is
the number of processors.
Similarly, there's a real limit to how much I/O traffic it makes sense
to schedule simultaneously. I need to do more research here - what
makes sense for my laptop with 1 magnetic hard drive is likely
different from a fast server with RAID on top of SSDs. For the
moment, I've chosen to limit I/O bound threads to 4.
The limit for _DEFAULT remains 100 - but we can solve this better if
we have e.g. a way for tasks to express dependencies, among other things.
But for now, with this patch, and OSTree modified to use
G_TASK_THREAD_KIND_IO for its rename()/link() threads, I am seeing
better performance.
https://bugzilla.gnome.org/show_bug.cgi?id=687223
9 files changed
