Got more questions? Find advice on: SQL | XML | Regular Expressions | Windows
in Search
Welcome to AspAdvice Sign in | Join | Help

Justin Lovell's Blog

Mood: The Grumpy Coder

Multi-threading... processes... splitting the atom... eish!

To start off, I am fairly new to threading. One of the recent “module” requirements and other future “module” requirements caused enough reason to start a small framework for time-slotted batch services. Basically, the idea is to have a thread running one 'process' in a certain reoccuring time slot (let's say for every one hour). If a 'process' is unable to complete itself in the allocated time slot, the process then creates a splitted-process to participiate over multiple threads. In practical terms, the 'process' just continously splits itself into more processes until n amount of threads are able to allow a 'process' to complete its task within the allocated time slot. Something that I would like to call “splitting the atom.”

We all know that when we split atoms, it is followed by a “kaaaboooommmm!” That is why I was so wary when designing this framework. It works very well under simulated unit tests (I've written about a day's worth of code for testing). Extremely well. I can bet my next pay check that this thing is able to scale without a problem... and that is the source of my cold-sweeted nightmares. Generally, things can go terribly wrong with multi-threading and the usual cause is because there are race conditions happening. :-/. That is not the case under my framework -- there is no race conditions but I can't get rid of the images that I might get that “kaaaboooom” and dreaming about the possible framework enhancements to unleash some restrictions that might be encountered.

Do you get multi-threading nightmares at night?

Published Wednesday, December 15, 2004 11:23 PM by jlovell

Comments

 

jlovell said:

Hey Justin,

I'm working on a multi-threaded application too right now, and yeah, it can get a little intense. Just be sure you don't go and spawn too many threads. It's bad that your process isn't finished yet.. it's worse when you are hosing the cpu by running 4-5 concurrent threads that absorb 100% cpu.. leaving the business to suffer until it's done.

if you want to bounce ideas back and forth, you know how to get ahold of me. Good luck with it, and let's not forget.. happy coding.

fyi: I talked to my bossman today, and he might be able to refer you to someone that will um.. fill your wallet a little.
December 16, 2004 8:32 AM
 

jlovell said:

Hi Kay,

There is a thread-watcher class in the framework that I built. It is not fine-tuned as of yet to allow the processes that take the most time to take priority (ie: have more threads dedicated to them than the other ones that jump slightly over the allocated time-slot). It also collapses the thread pool when it sees that two or more threads can be collected to form one process again that can fit in a time-slot.

In addition, these thread's default priority is below-normal... should I say that this will also fit on a dedicated box ;-). The unit tests were conducted on my development box at work and it showed 80% utilization under very heavy stress and only 35% utilization under the expected "heaviest" stress that this application will encounter.

In addition, this framework is designed so well that for a programmer to create a process, the person does not have to know anything about thread-safety to get the job done.
December 16, 2004 12:42 PM
 

jlovell said:

multi-threaded the I nightmares all get time :P
December 18, 2004 6:01 PM
Anonymous comments are disabled