ThreadPool And Large Processing Of WorkItem?
Oct 11, 2010
Been trying to implementing ThreadPool over a large and variable amount of data like the following:
public sub ProcessWorkItem(byval state as object)
Dim originalState as CustomStateInfo = CType(state, CustomStateInfo)
'Do stuff
[Code]....
This of course will bomb out if we have more than 64 items/threads being placed on the ThreadPool.
So, how would one process say x amount of items until there are no more items left to process?
View 30 Replies
ADVERTISEMENT
Mar 18, 2011
I just learned some basics of multi threading in VB.net recently as I came across processing a large List of records and inserting them into SQL database one by one.
I have code look like this:
Private Sub btnLoadNow_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnLoadNow.Click
Dim autoLoad1 As New Thread(AddressOf AutoLoad)
autoLoad1.Start()
[CODE]...
Itemlists is a global lists i retrieved from database, everything looks fine to me but when i run this program, I found that the threads are inserting one records 9 times into the database(I have 9 threads in total), which makes me think that maybe i need to specifically assign 1/9 of a list to each thread? Is there another way to do it which doesn't require the splitting of the list, if not , how do i split it then assign?
View 1 Replies
Jun 28, 2011
what is the best way of detecting if all the threads in the threadpool have finished working,
is there anyway to stop all the threads?
[URL]
View 8 Replies
Jun 4, 2011
I m trying to move all my drive's files into h drive firstly i have added all drives in threadpool for scanning and when scanning of any drive finish (Without waiting for other drive's complete ) it will move file's to h drive but it is not working as i wanted that's why i need two threadpool - one threadpool for scanning files and second threadpool for moving file's into drive's
View 11 Replies
May 21, 2009
I got a question on sql splitting a record into peaces.I got a big record like this and all of this in one piece of record:
"
5/21/2009 1:03:49 PM
Order has been placed
5/25/2009 1:45:49 PM
[code]....
This is used for making notes, so you can keep adding notes on it until it's done.And I know this is not the 'best' way of keeping these records, this is the way how it was kept until now.So now my manager comes to me and asks to make a different structure of processing these notes.Because it is stored this way I can only retrieve full record from the database, but what I need is a way how to take for example only the 'top line' of it.Example:5/25/2009 8:45:49 PM Confirmation send
View 4 Replies
May 22, 2012
I have spent way too much time trying to sort this little issue out. I have narrowed down the issue to the exact procedure that throws the error. Yes, I have used Google..
Try
Dim tempSource as String = Nothing
Console.WriteLine("Loading document...")
[code]....
I am loading a text file, that contains about 1100 lines, and each line is going to be processed with HTML Agility Pack. From what I can tell, when it runs "doc.loadhtml(richtextbox1)", it throws the error. I also have tried to load the file into a string, and load the string with "doc.loadhtml(thestring)". It doesn't make a difference, still errors.
Here are is a sample of how each line looks:
<Site Index="" Name="" Group="" PR="" />
<Site Index="" Name="" Group="" PR="" />
[code]....
I am using HTML Agility Pack, however the above is what is on every line, about 1100 lines! For testing, I have a smaller text file made of about 50 lines before I load up the 1100 line file ;) There aren't any HTML, HEAD, or BODY tags! They aren't needed for my parsing. I am using HTML Agility Pack because it is easy to parse elements with. I can grab each value easily from each line.
I am not sure if maybe the error is because it technically isn't HTML? Meaning since the loaded code doesn't have an HTML or BODY tag, that it errors? I wanted to get this question posted, and while I am waiting on some answers, I am going to parse the document another way. Just curious as to what the deal is and why HTML Agility Pack isn't working. More of a proof of concept then anything, for my own learning and knowledge.
Here is the error I get (btw the on the doc.load() line, is where it throws the exception):
Object reference not set to an instance of an object
Last Note: The routine is on a background thread. I have used multi-threading before, and have delegates created for deeper in the code.
View 1 Replies
Apr 19, 2010
I created a program that watches about 10 folders for different excel files. Using the filesystemwatcher with filters in place and correct error handling, it still misses some files once in a while (that's ok I guess).Anyways, I'm writing this because I added another form that has a filesystem watcher, when I tried to load the form I got a stack overflow error - an error that isn't debugged (a greenish line that you can only stop debugging).Am I going about this a wrong way? Is filesystemwatcher unreliable with large amounts of processing?
View 1 Replies
Jan 6, 2010
2008 Array Processing and Sequential File Processing
View 14 Replies
May 19, 2011
Im a college student and this is my first programming class, i am having extremely diffcult time with arrays and sequential file processing. Im not asking for the whole program, just something to get me started and hints along as i progress with it. Please help me get started on my program, its due in 2 days.
Here are the requirements....
1. The program must input any number of individual�s last and first names using two separate textboxes.
2. The array of names must be sorted by the last name using a bubble sort. You must code the sort.
3. The program must be able to recall the names from the file and add to the existing list of names so that more names can be added to, sorted, and written back to the file in sorted order.
4. Sort in ascending order: A to Z
5. All names must be displayed in a listbox in sorted order when the user wishes to see them.
6. In the listbox the last and first names must be separated by a comma and a space. Example: Smith, Mary
7. The user must be able to search for a name in the array, using the binary search algorithm. You must code the search.
8. Do not use module-level variables. Instead you may choose to use Static.
9. The program must be designed using Visual Basic 2008
View 3 Replies
Jun 9, 2011
I want to write a program to do Markov chain, but my states are quite large. First of all I calculate all the transition probabilities and revenues for all states(1381860 total states), and store in a multidimensional array. Public RevArr(0 To 9, 0 To 750, 0 To 282) As Long
After that the iteration of markov chain should use these as inputs to calculate the steady-state probabilities. But when I try to run the main code I got this error.Exception of type 'System.OutOfMemoryException' was thrown.
The following is the declaration of second array I add just another dimension for storing all the iterations, but I get this error. Dim stateprob(IT + 1, 0 To 9, 0 To 750, 0 To 282) As single
View 1 Replies
Jan 1, 2010
how this is written in VB.NET? This was an example I found on [URL]
ThreadPool.QueueUserWorkItem(delegate
{
var channelFactory = new ChannelFactory<ISimpleService>("*");
var simpleService = channelFactory.CreateChannel();
var asyncResult = simpleService.BeginGetGreeting("Daniel", null, null);
[code]....
View 3 Replies
Jun 26, 2011
i have this code which i run with queued 10,000, even when a max of 50 is set but the threadpool counts goes to high.
Public Sub DoWork(ByVal objItem As Object)
Dim objUrl as String = DirectCast(objItem, string)
Try
If objUrl Is Nothing Then
[code]....
EDIT:i use a this code to check the number of runing threads each second
Sub Timer1Tick(sender As Object, e As EventArgs)
tsslthreads.Text=string.Format ("Threads: {0}", Process.GetCurrentProcess().Threads.Count)
End Sub
View 3 Replies
Jun 12, 2009
I have a bunch of operations that need to be executed simultaneously. For this I'm using the thread pool to execute the tasks. I'd like to know if it's possible to consume some sort of Async Callback (similar to BackgroundWorker.DoWorkComplete) when each thread is finished. The reasoning behind this is that I have a windows service running that uses IPC to update a GUI and I need to know the time at which each specific thread finishes I've used background workers to do a set of tasks and they work perfectly, however, they take longer than I would like. I've timed the execution on my BGW method for a designated number of said tasks and it took 27 seconds, and the same set of tasks takes 4 seconds using the threadpool method. I've also tried using managed threads, to the same effect as the background workers.
View 5 Replies
Jan 11, 2009
I'm currently writing an application that is used to automate a certain task. Currently, it uses the threadpool (using up to 25 threads per the enduser's option). Each thread calls a function which will usually take about two minutes to finish, but it uses a While loop in order to repeat the process. This is my code to start the threads:
Private Sub ProcessAllItems()
For i As Integer = 1 To Me.numThreads.Value Step 1
Threading.ThreadPool.QueueUserWorkItem(New Threading.WaitCallback(AddressOf ProcessItem), i)
Next i
End Sub
The function "ProcessItem()" is the function that contains the While loop. The control "Me.numThreads" is a NumericUpDown control whose max value is set to 25. My question is...is this a good method to follow? To my knowledge after .NET Framework SP1 there is a max of 250 threads/CPU. However I've read the the threadpool is normally used for processing short-lived tasks. Should I be using background workers instead? If so, how could I start up to 25 background workers during runtime depending on the value of "Me.numThreads"?
View 14 Replies
Jul 2, 2009
currently i am using a threadpool that contain a number of thread. This number of thread can be different everytime, sometimes 5, sometimes 10, sometimes 15 and so on (multiple of 5). So now i have it working good by using for loop to queue the thread into threadpool. Now the problems is i do not know how to make the progress bar depending on my threadpool. I want my progress bar to complete only when all the thread are done. After searching google, i found that maybe it can be solve by using AutoResetEvent(false), but after i apply to my program, it still did not work, my program hang at xEvent.WaitOne() method and didnt proceed further, anyone can explain to me how to solve the problem? Or got any other ways? For your information, only 5 threads can be running at the same time as the requirement state so (so i set the setmaxthread to 5 already).
View 1 Replies
Mar 15, 2011
anybody familiar with these error? "There were not enough free threads in the ThreadPool object to complete the operation".im calling a web service in my windows application client.
View 16 Replies
Jun 5, 2010
whipped up this code real quick which uses a ThreadPool on Form Load to iterate through my subnet and tell me if there's a device communicating at each IP address.
some questions: Why am I able to update BOTH my label at the top AND the listbox, but I'm only using an .Invoke on the Listbox?How can I know how many threads are currently being used while that is executing?
What does the SyncLock do?I'm aware some people don't like the use of Application.DoEvents(). Is it ok to be using it in this instance or is there a better practice?For those wanting to try it out, drag a Label and a Listbox to a new form. Label is called "currentIPTextbox", Listbox is default name.
[Code]...
View 14 Replies
Feb 12, 2010
I prepare data to be processed, and saved to file. Since this process is slow, I want to delegate it to a parallel thread, so the main routine can end his work faster. The data is processed and saved to file by the "Sub ProcesarDatos", a routine on his instance of "FileProcessor class".
I call the code this way:
Dim
fp As
New
FileProcessor(DatosDeArchivo, Archivo & ".txt", Me)
ThreadPool.QueueUserWorkItem
(NewSystem.Threading.WaitCallback _
AddressOffp.ProcesarDatos))
View 2 Replies
Feb 14, 2012
I am using VB.NET 2008.Every now and then, ThreadPool.QueueUserWorkItem can take up to 500 msI checked the available thread, and it's not 0.What can cause ThreadPool.QueueUserWorkItem to take long sometimes, and how can I make it faster ?
Do While Not TerminateRisk
While qRiskSync.Count > 0
sItem = qRiskSync.Dequeue
[code]....
View 1 Replies
Mar 9, 2012
can somebody give ThreadPool example for application thats needs to get information to datagridview from thousands of sources? My application needs to work fast and without freezing and lag, but I cant figure out this threadpool thing. And would be nice if you could add this thing to example that when I set domainupdown value to 25 as example, it sets threads to 25. So its making work 25x faster. I haven't find any good example for this, only some C# examples but they haven't work after i converted those to vb.net.
View 1 Replies
Nov 20, 2009
I have a app that gathers and store updated data from a list of websites. My app use threadpool and each thread visits a site within the list and gathers the data. I want to create a stop button to stop/ prevent threadpool from running once it is clicked. How do I go about doing this?
View 1 Replies
Jul 1, 2009
currently i am using a threadpool that contain a number of thread. This number of thread can be different everytime, sometimes 5, sometimes 10, sometimes 15 and so on (multiple of 5). So now i have it working good by using for loop to queue the thread into threadpool. Now the problems is i do not know how to make the progress bar depending on my threadpool. I want my progress bar to complete only when all the thread are done. After searching google, i found that maybe it can be solve by using AutoResetEvent(false), but after i apply to my program, it still did not work, my program hang at xEvent.WaitOne() method and didnt proceed further, anyone can explain to me how to solve the problem? Or got any other ways? For your information, only 5 threads can be running at the same time as the requirement state so (so i set the setmaxthread to 5 already).
View 2 Replies
Sep 15, 2009
I want to open a VB.net form using a ThreadPool with a maximum of 10 threads. I need to do this because, the form should open based on a live event and i need atmost 10 forms on the desktop at a time. If i open the form on every live event without using a ThreadPool, the desktop will be floaded with many forms. I am using QueueUserWorkItem method of threadpool to queue a method which opens the form. In this queued method, there is form1.show() statement called. But after displaying the form, this method finishes execution and again we have 10 threads available in the system. I somehow want to either keep this method running so that 1 thread is occupied or i want to keep the thread alive till the form1 is closed by the user. I want the 11th form to appear on the desktop, only when any of the existing forms out of the 10 (already on desktop) is closed.
View 2 Replies
Jun 4, 2009
When using teh system.threading.timer or threading.threadpool. queueuserworkitem, you are able to pass an object.
But wouldn't the garbage collector destroy the object even before the callback can be executed?[code]...
View 2 Replies
Feb 7, 2010
Every time that I want to make a thread in the ThreadPool I make a stupid little function like Worker_O below.[code]...
Is there a way in VB .net to cast from Sub(i as integer) to Sub(o as object) without making Worker_O? Worker_O is ugly to me.
View 1 Replies
Aug 14, 2009
I have a task that needs to be executed many times in a thread, so as to keep my GUI responsive. Right now I'm using the ThreadPool to accomplish the tasks, and it works perfectly. I get all the data I expect back from the threads, life's good. When those tasks are finished, I then fire off another set of threads to accomplish another set of tasks, in the same fashion. The work method of these threads uses a call to RegistryKey.OpenRemoteBaseKey, which does exactly that, opens a remote registry key. The problem is: if the user running the application does not have permission to open the remote registry the call will throw an IOException, but I expect this behaviour because the user does not have permission to do it. However, the call itself can take FOREVER.
I wrote the application initially to use delegate methods and used the WaitHandle.WaitOne() method on AutoResetEvents in a state object I passed to each delegate. I tried the WaitOne method using a timeout, which worked fine until I realized that even though the WaitOne timed out, the thread was still running. This is a problem because (from what I understand) processes are only given a certain number of ThreadPool threads and, once used, they must be finished in order to be released back to the threadpool. In my work method, I also have a variable that can be signalled when the thread should be stopped, but that (obviously) only works on code that I've written.how can I cancel a thread that's stuck on a .NET method? Is there some sort of garbage collection method I can call if I give it a handle?
View 18 Replies
May 5, 2009
The error is:
Method 'Private Sub ProcessToolWork()'
does not have a signature compatible
with delegate 'Delegate Sub
[code].....
View 2 Replies
Sep 13, 2011
Learning about threads at the moment, barely managing to wrap my head around it, anyway I was wondering how you would go about making threads run sequentially from threadpool if you added them with
threadpool.queueuserworkitem()
In my code currently I know it is not running sequentially as I set it up to call a function and print a number each call. I gather this could be achieved with using synch lock but not quite sure how.
View 5 Replies
Feb 29, 2012
I am doing one web scraper which loops user set list of urls. Lists can be huge and that's why I need Threadpool for it.
My code so far (Scrape code missing)
Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
Dim ThreadingEvents(20) As ManualResetEvent
Dim iCounter As Integer = 0
For Each item As Object In ListBox1.Items
[Code] .....
View 4 Replies
Jan 16, 2012
I have a code which adds each line of a textfile to a List(Of String).I have around 1.5 million entries in the textfile. It adds it to the list in like 1 second. However, I'm trying to add all the users to a threadpool to do a certain function. So I try:
For i as integer = 0 to list.count - 1
The list has like 1.5 million entries so the GUI freezes untill it all finishes in like 5 minutes. How can I make it go faster and not freeze the GUI?
View 6 Replies
