Multi-threading Processing - Processing A Large List Of Records And Inserting Them Into SQL Database One By One
Mar 18, 2011
I just learned some basics of multi threading in VB.net recently as I came across processing a large List of records and inserting them into SQL database one by one.
I have code look like this:
Private Sub btnLoadNow_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnLoadNow.Click
Dim autoLoad1 As New Thread(AddressOf AutoLoad)
autoLoad1.Start()
[CODE]...
Itemlists is a global lists i retrieved from database, everything looks fine to me but when i run this program, I found that the threads are inserting one records 9 times into the database(I have 9 threads in total), which makes me think that maybe i need to specifically assign 1/9 of a list to each thread? Is there another way to do it which doesn't require the splitting of the list, if not , how do i split it then assign?
Im a college student and this is my first programming class, i am having extremely diffcult time with arrays and sequential file processing. Im not asking for the whole program, just something to get me started and hints along as i progress with it. Please help me get started on my program, its due in 2 days.
Here are the requirements....
1. The program must input any number of individual�s last and first names using two separate textboxes. 2. The array of names must be sorted by the last name using a bubble sort. You must code the sort. 3. The program must be able to recall the names from the file and add to the existing list of names so that more names can be added to, sorted, and written back to the file in sorted order. 4. Sort in ascending order: A to Z 5. All names must be displayed in a listbox in sorted order when the user wishes to see them. 6. In the listbox the last and first names must be separated by a comma and a space. Example: Smith, Mary 7. The user must be able to search for a name in the array, using the binary search algorithm. You must code the search. 8. Do not use module-level variables. Instead you may choose to use Static. 9. The program must be designed using Visual Basic 2008
I got a question on sql splitting a record into peaces.I got a big record like this and all of this in one piece of record:
" 5/21/2009 1:03:49 PM Order has been placed 5/25/2009 1:45:49 PM
[code]....
This is used for making notes, so you can keep adding notes on it until it's done.And I know this is not the 'best' way of keeping these records, this is the way how it was kept until now.So now my manager comes to me and asks to make a different structure of processing these notes.Because it is stored this way I can only retrieve full record from the database, but what I need is a way how to take for example only the 'top line' of it.Example:5/25/2009 8:45:49 PM Confirmation send
I have spent way too much time trying to sort this little issue out. I have narrowed down the issue to the exact procedure that throws the error. Yes, I have used Google..
Try Dim tempSource as String = Nothing Console.WriteLine("Loading document...")
[code]....
I am loading a text file, that contains about 1100 lines, and each line is going to be processed with HTML Agility Pack. From what I can tell, when it runs "doc.loadhtml(richtextbox1)", it throws the error. I also have tried to load the file into a string, and load the string with "doc.loadhtml(thestring)". It doesn't make a difference, still errors.
I am using HTML Agility Pack, however the above is what is on every line, about 1100 lines! For testing, I have a smaller text file made of about 50 lines before I load up the 1100 line file ;) There aren't any HTML, HEAD, or BODY tags! They aren't needed for my parsing. I am using HTML Agility Pack because it is easy to parse elements with. I can grab each value easily from each line.
I am not sure if maybe the error is because it technically isn't HTML? Meaning since the loaded code doesn't have an HTML or BODY tag, that it errors? I wanted to get this question posted, and while I am waiting on some answers, I am going to parse the document another way. Just curious as to what the deal is and why HTML Agility Pack isn't working. More of a proof of concept then anything, for my own learning and knowledge.
Here is the error I get (btw the on the doc.load() line, is where it throws the exception):
Object reference not set to an instance of an object
Last Note: The routine is on a background thread. I have used multi-threading before, and have delegates created for deeper in the code.
I created a program that watches about 10 folders for different excel files. Using the filesystemwatcher with filters in place and correct error handling, it still misses some files once in a while (that's ok I guess).Anyways, I'm writing this because I added another form that has a filesystem watcher, when I tried to load the form I got a stack overflow error - an error that isn't debugged (a greenish line that you can only stop debugging).Am I going about this a wrong way? Is filesystemwatcher unreliable with large amounts of processing?
I am creating a simple Website where users will update SQL database. The website works fine however, new records in the SQL database records needs to appended every night from a text file. What is best way to append records in the background every night? have created a stand alone vbscript to append the recoeds to the table in SQL Database hoping to schedule it to run every night but the problem I am facing is when users are using the Website my vbscript fails with the message "The file is being used by
Basicly I'm downloading XML files with the Webclient class for processing and caching in a local database. There are three different categories of data, each contains 1 to n numbers of XML files. To retrieve the XML file I ask the server with following parameters in the URL:
I typically dont use a dataset in a case of new data insertion to the database. But then again, I will run in a problem the weather the inserting data already exsit in the database. Therefore i need to perform a search first. However, if i follow this it will take slow down the application (if the table has 1 million records). One way i could optmimize this is to load the whole table in to a dataset then the application doesnt have to go to a back end trip nwo it can search inthe dataset. Is this the standard way for inserting records?
I am trying to insert records in an Access database using VB.NET 2008 using the following code:[code]I get this error Syntax error (Missing operator) in query expression when I try to insert any record that has apostrophes in the name.I have tried everything I can think of to get these types of names intserted into the database with no luck.
I created tab controls to seperate the controls till this point everything is OK. but the problem is I can't insert records into database from the tabs controls. but when I enter in all of tabs the insert statment is running.
At the moment, my project has a method do_job(jobid,userid)
This method take a minute or two, and then redirects to another page. This has been rewritten so that it now runs in the background and redirects to another page whilst the thread which do_job is on continues. - This works fine. In a button_click event in a default.aspx page :
Dim d as New do_job_delegate(AddressOf do_job) d.BeginInvoke(jobid,userid, New AsyncCallback(AddressOf Callback),d) Response.redirect("results.aspx")
[Code]....
So what i want to happen is - the longest time that do_job takes is the longest time of the longest job (30 seconds), rather than - 30 seconds x number of jobs = time it takes at the moment.
As i'm already using a delegate to detact the process from the ui, can anyone provide any advice/best practice on how i should rearrange my do_job method so that it can handle concurrent tasks to further boost the overall speed.
If vb application (windows form app) is calling a store procedure and it takes 5-10 mins to SP to finish and meanwhile the application is (forced to) close,
what will happend at the SQL side?
Will the proc completely run even though the app is not running anymore?
I want to run my vb.net 2003 application from command line and accepting only one parameter(argument)....... BUT it should run without GUI, only a function from my code would get run in background..
My program runs with no errors, but the actual .accdb file does not reflect changes when I make them in the running program. Also when I fill in the text boxes and click the "Add New Item" Button, the "ID" text box displays a -1. "ID" is the primary key in my Access Database.
Imports System.Data.OleDb Public Class addMemberForm Inherits System.Windows.Forms.Form
I am looking for a solution to implement Credit Card processing to a POS system I have written. I have been coding VB for about 7 years but never had a need to process credit cards.
I haven't seen a related thread (yet)I have a VB windows application that uses a datagrid to display a text column and a checkbox column. Only one row in the display can have a checkbox entry = true. If a checkbox is selected, the app is to search the rows and all other checkbox entries are to be set to false.My event handler is: Private Sub dgMember_CellContentClick(ByVal sender As System.Object, ByVal e As ystem.Windows.Forms.DataGridViewCellEventArgs) Handles dgMember.CellValueChangedI first had to have this handler code check a flag and exit if true (when initializing the display) because it was called for each row during initialization.
I have a form with several buttons on it.When a button is pressed,it will execute some remoting proxy method on the server. There is some delay in the process before there is a result returned from server .Currently,I tried to disable the button in the _Click event, hoping that a 2nd, 3rd and the following clicks would not be registered.Unfortunately, those input from the mouse clicks were queued by Windows.
So once the _Click event has finished running, it will immediately get executed again due to the earlier clicking.I tried to create a form (Notice) to display some status message, hoping that clicks won't be registered on the button. Despite setting the form to maximize and on top of the caller form, the clicks still get registered.Then I thought of using .ShowDialog(). Unfortunately, it halts the processing until I close the form. So I did something more complicated by passing in the method into Notice form, so that it would execute the _Click method via an address pointer.Unfortunately, after all these coding. The buttons still get registered with the mouse clicks.So now, is there any way to overcome this clicking queue that Windows registered?
i have 10 checkboxes...i want a short way to check if checkbox1.checked=true and check box3.checked=true.... blah blah blah and so on to do codes...i was using long typeing code like:
if checkbox1.checked=true and checkbox2.checked=false and checkbox3.checked=true and checkbox4.checked=true and checkbox5.checked=true and checkbox1.checked=true.....etc then msgbox"done" end if
i am using vb.net to program and process some files.I pick files from one folder process them and once i process them successfully i should store them in a different folder. when i do fi.move(destination folder) i get an error message saying "THIS FILE IS USED BY ANOTHER PROCESS" below is my code
I'm thinking of making use of online technology to do some validation and processing for my program but I'm unsure of its feasibility. I need to create a program whereby the client can retrieve the data/settings from website say http:[url].... .Another example would be allowing the program to run until end of December by making use of online time service to check the date so that user can't change any settings from their computer.
While discussing about retrieving the data either through html static site or webservices, another thing I'm concern is transmitting the data from client side to my side.I wonder what is the best and free way to do that? As I'm thinking of using a free smtp gmail to transmit data but I'm worried about the security issue as credentials would be stored inside the program and also firewall might block data from sending out. Another choice would be sending a POST request, but I do not have any server or is there any free server that allows user to post some small bytes of data?
