C# - Avoiding Deadlocks And TimeOuts When Processing Huge Data?
Oct 14, 2010
I have code in an ASP.NET form that needs to, depending on user entry create messages in the database. We are speaking of potentially thousands of db entries. How do I protect against deadlocks, I mean apart from using Transactions and setting IsolationLevel to Serializable, as well as using WITH(NOLOCK) statement on my select statements since I don't mind a dirty read.
[Code]...
View 1 Replies
ADVERTISEMENT
Jun 5, 2011
I have a huge amount of data in flat-file format that I would like to validate with Visual Basic 2010. Basically the data is a text file with 300 comma separated fields and 500,000 records.
Any tips, sample code or examples? Are any .NET classes specifically geared for data validation?
As I have different validation rules for each field (some fields are dates, some are currency, etc.) I would like to store the rules away from the code (e.g. in an SQL database table). Has anyone seen this done before?
View 1 Replies
Nov 16, 2010
im using vs2010 and SQL server 2005. what is the best and fastest way to load large volume of data into treeview
View 6 Replies
Sep 16, 2011
If dr("iname").ToString = DataGridView1.item(0).cells.item(0).valueThen Then
MsgBox("Similar items are not allowed.", MsgBoxStyle.Critical)
Exit Sub
End If
I need to avoid displaying similar data from my mysql database table iname to the gridview, but I can't identify the right syntax/code for that?
View 1 Replies
May 19, 2012
web method:
[Code]...
with this condition(PACK_DATE > "1388/11/07" 366 records) everything works well.but when i change date to 1388/11/06 1260 records or 1388/11/05 5460 records or ... following error occurs:
[Code]...
View 1 Replies
Mar 4, 2011
My app is a fairly simple and im sure with the right knowledge this would take 2 minutes.. but sadly iv been at it for hours. So my scenario here is a huge network with 20+ file servers all with data spread out across them it can take hours to find the folder required.
Hence this app.. so it consist of a input box, progress bar and a button, using FileSystemObject I plan to interrogate a directory to see if a folder exists (folder specified by user input) however it need to check multiple directories, in my old app I used a list box which wrote to a text file allowing the user to add additional directories then using if statements proceeded through the servers.. however they have added that much that my IF's have ran out really because i know it should have been a loop
Imports System
Imports System.IO
Public Class Frm_main
Private Const FILE_NAME As String = "C:directories.txt"
Dim objReader As New System.IO.StreamReader(FILE_NAME)
Dim Lines() As String = IO.File.ReadAllLines(FILE_NAME)
Dim Dir As String
[CODE]...................
View 1 Replies
Mar 18, 2011
I just learned some basics of multi threading in VB.net recently as I came across processing a large List of records and inserting them into SQL database one by one.
I have code look like this:
Private Sub btnLoadNow_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnLoadNow.Click
Dim autoLoad1 As New Thread(AddressOf AutoLoad)
autoLoad1.Start()
[CODE]...
Itemlists is a global lists i retrieved from database, everything looks fine to me but when i run this program, I found that the threads are inserting one records 9 times into the database(I have 9 threads in total), which makes me think that maybe i need to specifically assign 1/9 of a list to each thread? Is there another way to do it which doesn't require the splitting of the list, if not , how do i split it then assign?
View 1 Replies
Jan 6, 2010
2008 Array Processing and Sequential File Processing
View 14 Replies
May 19, 2011
Im a college student and this is my first programming class, i am having extremely diffcult time with arrays and sequential file processing. Im not asking for the whole program, just something to get me started and hints along as i progress with it. Please help me get started on my program, its due in 2 days.
Here are the requirements....
1. The program must input any number of individual�s last and first names using two separate textboxes.
2. The array of names must be sorted by the last name using a bubble sort. You must code the sort.
3. The program must be able to recall the names from the file and add to the existing list of names so that more names can be added to, sorted, and written back to the file in sorted order.
4. Sort in ascending order: A to Z
5. All names must be displayed in a listbox in sorted order when the user wishes to see them.
6. In the listbox the last and first names must be separated by a comma and a space. Example: Smith, Mary
7. The user must be able to search for a name in the array, using the binary search algorithm. You must code the search.
8. Do not use module-level variables. Instead you may choose to use Static.
9. The program must be designed using Visual Basic 2008
View 3 Replies
Mar 31, 2011
If vb application (windows form app) is calling a store procedure and it takes 5-10 mins to SP to finish and meanwhile the application is (forced to) close,
what will happend at the SQL side?
Will the proc completely run even though the app is not running anymore?
View 1 Replies
Dec 9, 2009
The following is my connectionstring: server=(local);database=JnJ3;uid=sa;pwd=sa May I know how can I avoid timeout expired because I keep on receiving error message when deal with huge amount of data. The timeout period elapsed prior to completion of the operation or the server is not responding.
View 1 Replies
Mar 26, 2010
Is there an easy way of trapping the enter key in a textbox to process the data, instead of having a button beside it?
View 2 Replies
Feb 27, 2009
I have a text file that contained about 50,000 lines of data which are all the same format as below
0000710950,INFEED NAME,40,INFEED TIME,0,ENTRY,55649,MCS,[code]....
What I would like to do is read this data in as fast as possible either line by line and processing the data as I go by line. Or read the whole lot in and THEN process it, which woulkd be the fastest ? I used to do this in Excel but want to transfer it to VB and don't know really how to get started I have the following but would like to know how to get the columns into a named array and then how to itterate through array.
Dim myLines() As String = IO.File.ReadAllLines("C:SYNC_DATA�0 - LGW�5 - Archived�0 - IST Logs2008istlog20090224.txt")
View 4 Replies
May 3, 2011
I'm using HTMLAgilityPack in a parser that I have up on a server, but I'm having issues with one of the websites that I'm parsing: Every day around 6am they tend to shut down their servers for maintenance, which throws off the Load() method for HTMLWeb, and makes my app crash. Do any of you guys have a more secure way of loading a website into HTMLAgilityPack, or maybe some way to do error checking in C# to prevent my app from crashing? (my c# is a little rusty). Here is my code right now:
HtmlWeb webGet = new HtmlWeb();
HtmlDocument document = webGet.Load(dealsiteLink); //The Load() method here stalls the program because it takes 1 or 2 minutes before it realizes the website is down
View 2 Replies
Apr 20, 2011
The current version of my application was using the WebClient object to download files from the internet and all was working well until some users reported that they were getting server timeouts. I looked into this issue and decided to move to the WebRequest object as it allows me to specify a Timeout value. The problem is, when using it synchronously, if I specify a timeout value of -1, I am worried that in the event of some weird server anamoly, my application would hang as it would be waiting for a response that isn't coming. So then I thought of using the WebRequest ansynchronously.
[Code]....
View 8 Replies
May 27, 2011
Let's say an application needs to persist mostly immutable data. Let's say the application's data files can grow fairly large. How would you implement a simple transaction processing so that changes are either successfully committed or rolled back?
The only requirement is to use built-in .Net libraries. (no databases, no third party libraries)
Here's my current solution. The data file will have three main sections: (1) header, (2) data, (3) index. For now, the header section simply contains an offset to the start of the index section. The data section will store mostly immutable data. The index section will contain serialized objects with enough information to access the data section.
When the application starts, it only needs to deserialize objects from index section. Anything from the data section can be fetched as needed. As the application gets ready to write changes, it makes a temporary copy of the original header and index sections. These would be small relative to the data section. New data would then be appended to the original data section. Once all data has been written, in memory objects would be serialized to the end of the file to become the new index section.
If all this goes without a hitch, the temporary file can be deleted. If something goes wrong, the temporary file can be used to restore the data file to its original state.
Will this work or is there a more elegant approach?
View 1 Replies
Jul 28, 2009
I'm writing a web crawler for a specific site. The application is a VB.Net Windows Forms application that is not using multiple threads - each web request is consecutive. However, after ten successful page retrievals every successive request times out. I have reviewed the similar questions already posted here on SO, and have implemented the recommended techniques into my GetPage routine, shown below:
[Code]...
View 4 Replies
Jan 10, 2012
In an asp.net 2.0 application I maintain, we were experiencing issues with transaction aborted errors (timeout). Code that fails seems to cause a timeout, then the page logging functionality that uses transactionscope (default constructor) fails (but not always). The timeout is set to 2 minutes. Some example code that is similar to what we have in our app is below:[code].....
From what I've seen and read, my guess is that since the using block is never reached, the transaction times out.Then the logging code (done with any page request) tries to enlist in the existing transaction, which has timed out and results in a transaction aborted error (as soon as the constructor is called). Is this assumption correct? Why would only some requests fail going forward and not all (assuming they all use transactionscope)? how does connection pooling play into this? If user A hits the bad code, would user B then be potentially affected by this? This is the behavior we've seen. If not, what else might cause this? I've been to MSDN, but I can't find anything that really clicks with me as to what is happening and why.Here is the relevant portion of the connection string:
Enlist=true;Pooling=true;Connection Lifetime=20;Max Pool Size=25;Min Pool Size=5
FYI. Not sure if this is relevant, but the app uses an Oracle 11g database with the EntLib database factory pattern.
View 1 Replies
Feb 19, 2009
I'm running some unit tests for a large VB.Net application I help maintain and am running into problems with tests timing out. The test run is set up so that there's no timeout limit on the overall test run, but a limit of 2 minutes per individual test. However, a couple of the tests take a lot longer than this to run, so we tried adding the timeout attribute to the test's declaration, i.e
<TestMethod(), Timeout(15 * 60 * 1000)> Public Sub DoSomething()
for a 15 minute timeout. However, the 2 minute timeout set in the test run properties seems to be taking precedence, even though it says that the timeout attribute should override it.
View 1 Replies
Jan 18, 2011
I have been using the following Stream Reader code for over a year now to pull information from 3 websites w/ no issues. Recently my users are getting massive amounts of "Timeout" errors. I have opened a ticket w/ the IT department but they have not found any issues (we are experiencing off/on issues w/ our web traffic speeds in our department). Per the code I have the timeout set to 20 seconds but the Timeout error occurs in only a few seconds from the System.Net.WebException.
Imports System.Text.RegularExpressions
Imports System.Net
Imports System.IO
[code]....
View 5 Replies
Jul 31, 2009
I have been writing code fore a rather large project in the past mounts And Im beginning to think about arcitecture of the program... Its just a complete mess. I dont know how to tackle so much code..Its beginning to be impossible to debug.Basically Im doing the following prosidures. Importing several a Excel files and getting the rfrequency data from these files..Displaying them in a litview and a datatable... so you are able to edit them..Doing some calculation and exporting the results back to a xls file.Problem is now that im going to add more functionallity to the program,. What im woundering about is, is there some good unofficial guides on how to program with OOP?
View 11 Replies
Sep 21, 2009
I've been messing around with prime numbers for a while now, but I now want to use massive numbers, say a few million digits in length. Is there a way of storing and performing mathematical functions with such large numbers? Is it even possible on my home computer? I read on another old thread, something about a big number library, but the links he gave are now dead.
View 4 Replies
Mar 29, 2010
I have a program that tracks packages. Shipment and package entities are persisted in SQL database, and are displayed in a WPF view. Upon initial retrieval of the records, there is a noticeable pause before displaying the new shipments view, and I have not even implemented the code that counts shipments that are overdue/active yet (which will necessitate a tracking check via web service, and a lot of time).I have built this with the Ocean framework, and all appears to be doing well, except when I first started my foray into multi-threading. It broke, and it appeared to break something in Ocean... Here is what I did:
Private QueryThread As New System.Threading.Thread(AddressOf GetShipments)
Public Sub New()
' Insert code required on object creation below this point.
[code].....
View 1 Replies
Aug 1, 2008
Ok New thread - god I hate laptops sometimes - I just 'lost' a huge post in the editor due
View 1 Replies
Jun 15, 2010
I have a huge .txt file which contains X & Y data seperated by serialization. Each serialization contains 1K x and y data points, the text file is about lines 1,340,000 long. Sometimes I need to go into the .txt file and find certain sets of data so I can plot and overlay them which gets tedious. I am going to write an application that I can just write the serialization into a text box on a form and and it will seach all of the txt files for the right data, put them into arrays, and then load them up into excel. My question is how would I search the .txt file for the correct serialization? Would I first read it into an array and then try and split it that way, or is there a way to just open the .txt file and search for the serialization without putting it into an array first?
View 3 Replies
Mar 8, 2010
In .NET, the Generics Lists have a sort function that accepts IComparer or Comparison. I'd like to sort just part of a list. Hopefully I can specify the start index, count of elements to sort, and a lambda function. It looks like you can only use lambda functions to do this if you're sorting the entire list. Is that right or did I miss something?
View 4 Replies
Oct 5, 2009
My program uses a wmp control to play sound, but I am not sure why it is listed in Windows 7's Sound mixer and has a huge icon. How do I change this?
View 7 Replies
Apr 21, 2012
Is there a way I can store HUGE strings without using settings? Or storing it outside the program?
View 1 Replies
Dec 18, 2011
I have a past experience with visual Foxpro and SQL. Now, I need to build a database with multiple data files (say 100 files), each data file with millions of records, each record with dozens of fields.Then, each data file is linked with other files by using different fields. Primary data is in English. Then there would be a translation of the English data is in another language. Primary data in English is stored in separate files whereas the translated data is stored in separate data files.
View 4 Replies
Sep 22, 2009
I've been messing around with prime numbers for a while now, but I now want to use massive numbers, say a few million digits in length. Is there a way of storing and performing mathematical functions with such large numbers? Is it even possible on my home computer? I read on another old thread, something about a big number library, but the links he gave are now dead.
View 1 Replies