.net - Server Halts Due To Large StringBuilder Instance In Memory?
Aug 2, 2010
So, I have this code that grabs a bunch of data from the database, and performs a bunch of calculations on that data. However, this is not what is causing the halt. The halt is coming in when I take all that "final" data that has been prepared, and go to write that into a text file.
Each line in the text file is being created from data that has had calculations performed on it. Each line "corresponds" to a record from the database. A user will have to perform this file export usually on about 30-40 thousand records at a time. Now, even though I am using a StringBuilder, this still halts because of the pure mass amount of records.
What I do currently in the code is create one StringBuilder, append all the data (after calcs) to it, convert it to a String, add it to a List(Of String) to be returned from the Function, and then clear the StringBuilder, and do the same for the next row, and I do this until each row has been created and appended, converted, and then inserted into the List.
Then, another class receives this List(Of String) and takes each String and eventually appends it to a file.
Does anyone have an idea of how to improve this code and make it so it doesn't halt the server every time? The reason it wasn't found out until now is because when I tested it locally I was testing it on only 1-50 records at a time.
I am facing an amazing problem in my application. On my Developement server when I am running my application, then there is one function where I am trying to put whole stringbuilder contents in a string object. This Function is used for paging the reports.For that I am using a method of Stringbuilder.tostring() and when I chek the value of this statement (stringbuilder.tostring()) it shows no such interface is supported, which later on throws exception of system is out of memory. Here i am using .net 2.0 framework and sql server 2005 database.But when I run the application on stagging server then there is no such exception thrown.[code]
I've found the following code in my boss's project:
Dim strBuilder As New System.Text.StringBuilder("", 1000000)
Before I call him out on it, I'd like to confirm whether this line actually sets a megabyte (or two megabytes in Unicode?) of memory aside for that one stringbuilder?
I am using .net 2.0 and I am getting an exception of system is out of memory. My existing function takes datatable as input parameter which contains little bit large data (eg. more than 35000 rows) and in that function I am appending some xml node data and atlast I am returning the stringbuilder as a object.
Ok so i have an jpg image of around 8.10mbs at around 13424 x 11344 pixels, i need to load the whole image in to a scrolling picturebox, is they anyway to do this without using so much memory for example like what google maps does n cuts the image down and some how stitches it all back to gether?
I have a problem with a memory leak in a very large ASP.NET application. After about 24 hours of usage (sometimes a lot more) an OutOfMemory exception is thrown. Therefore I am trying to understand how managed memory works in .NET. When does an ASP.NET application throw an OutOfMemory exception? The server has a lot of RAM and there is always plenty of memory left when an OutOfMemory exception is thrown. I understand the difference between virtual memory and physical memory. How much memory does the W3WP process have to consume before an OutOfMemory exception is thrown? Is there a setting somewhere e.g. in the Machine.Config file? The ASP.NET process is never recycled.
I need to improve memory performance on my application and I could see that I have problems with memory fragmentation.I've read an interesting article on large objects from Andrew Hunter of Red Gate, and one of the solutions he recommends isow do I implement his suggestion in my code?My program has a very complex form (with an object that leaves residual memory every time it opens. I found a complex list that may be the culprit, and I'd like to implement his suggestion to see if it fixes the issue.
I'm loading an image, then getting size and such info from it, and loading the image into a picture box. Now it works and all, but I noticed it uses a large amount of memory. I'm doing: img = Image.FromFile(file)Where I load the image, 'file' being just a string containing path to an image on the hd. After that, I get the size and frame count (if any), and load it into a picture box: The image I've been testing with, is a 6600x5100 3.5MB JPEG, which can be downloaded from url...Watching task manager, when the app loads the image, it takes up about 104MB of memory. I've tried doing img.Dispose after the above line, but it brings up an error. If I put a button on the form with img.Dispose in it, and click it after app has loaded, it drops down to about 6MB of memory used. Loading the image directly into the picture box using .Load or .LoadAsync, after disposing img beforehand, still ends up with a huge amount of memory used.
So I'm wondering if theres a way to check if the picture box has fully loaded the img, so then I can Dispose the img and free up the memory. Or maybe a better way all together of loading an image into a picture box, and/or getting size & frame count, without having to use up that amount of memory before freeing.
I am retrieving a large BLOB and keep getting an Out Of Memory Exception. It occurs when I use a SqlDataReader to either check if the column is NULL, or if I try to read it and assign it to a Byte type of parameter, or if I use Response.BinaryWrite.
I need some guidance on a little program I'm making (one that I thought would take a couple of hours from start to finish, it's been days now ...) that determines all of the proper divisors of any given number up to the maximum value of a UInt32.
I recently wrote a program that collects information from the network can displays it in a DGV for the user to see. I tested this on networks where I would end up with several thousand items brought back and all worked fine, very little memory increase and once the search had finished the memory usage went back to pretty much what it was before the search, so I assume I have no major memory leaks that I should be concerned about (I'm careful to always dispose of things and clear large variables where possible).
However, a new company has now started using my application and they have several hundred thousand items that my program will find. They reported that they leave the program running its search for a few hours and most of the time it crashes with an Out Of Memory exception. Now I did have my app do some caching of certain information to avoid querying the network more than was necessary so I thought well maybe this is what is causing it but I added an option to turn that off and they say it still happens. So the only thing I can see it being is simply the amount of data that the program is finding and having to store in memory.once my application has gathered information about the items it stores them in a List(Of MyItemClass) and then when the search is complete the items are added to the DGV. So I'm thinking maybe its just the size of this List(Of MyItemClass) that is simply getting too large. Bear in mind that each instance of MyItemClass in this List can have quite a lot of information in it as it has several properties that are List(Of String) that may contain a few thousand strings in some scenarios.How do you deal with such large amounts of data without running out of memory?I'm assuming the only option I have got is to 'page' some of the data to a temporary file on disk once I get over a certain number of items in the List(Of MyItemClass)?
I have a user interface app that allows you to drag/drop in files - and it turns them into HttpListener request POST's to another EXECUTABLE that I have running - let's call it BACKEND1. BACKEND1 will be running on a SERVER when this gets ready for production use. Lots and lots of users running the UI - all dragging in files - all getting POST'ed to BACKEND1 - with the file included as a memory stream in the POST. BACKEND1 responds to the UI with a "sequential" file number assigned to the file centrally on the server.
On the SERVER is also running another EXECUTABLE - let's call it BACKEND2. It's job is to work with the DRAG'd in files. Both BACKEND1 and BACKEND2 sit on the same SERVER. At the moment I am writing the memory stream to DISK in BACKEND1 - and POSTing the FILENAME to BACKEND2 for it to work on. Would it be better to not write the file in BACKEND1 but instead include it in the POST to BACKEND2 - where it can be written to DISK? Is having LARGE HTTP REQUEST's a burden when sent between two EXECUTABLES on the same machine?
My primary goal is to have a really available BACKEND1 talking the the UI instances running out in the world. I'm concerned that doing the file i/o in BACKEND1 is "expensive" - and I don't care about the performance of BACKEND2 as much. just looking for opinions. I guess ultimately I'll have to bench mark this myself in a production environment to get a real answer.
I seem to be having some issues with my code here. The sub "CheckSettings" is actually run off of a system timer, so that every 5 seconds it calls CheckSettings (actually, i set it to every 1 second for debugging, so i can see the leak a little more clearly by monitoring the process). CheckSettings basically checks to ensure there isn't already an instance of the thread running, and if there is, it checks to ensure that the instance has completed running ... once it has the go ahead, it creates a new thread and runs again.
code:
This code is creating a very large memory leak. After a few minutes of running, my application goes from a memory footprint of about 45,000 K to about 1,000,000 K. Quite the leak. I read some articles on the managed versus unmanaged memory etc, and I was careful to free the objects etc.
I have a user interface app that allows you to drag/drop in files - and it turns them into HttpListener request POST's to another EXECUTABLE that I have running - let's call it BACKEND1.BACKEND1 will be running on a SERVER when this gets ready for production use.Lots and lots of users running the UI - all dragging in files - all getting POST'ed to BACKEND1 - with the file included as a memory stream in the POST. BACKEND1 responds to the UI with a "sequential" file number assigned to the file centrally on the server.
On the SERVER is also running another EXECUTABLE - let's call it BACKEND2. It's job is to work with the DRAG'd in files.Both BACKEND1 and BACKEND2 sit on the same SERVER. At the moment I am writing the memory stream to DISK in BACKEND1 - and POSTing the FILENAME to BACKEND2 for it to work on.Would it be better to not write the file in BACKEND1 but instead include it in the POST to BACKEND2 - where it can be written to DISK?
Is having LARGE HTTP REQUEST's a burden when sent between two EXECUTABLES on the same machine?My primary goal is to have a really available BACKEND1 talking the the UI instances running out in the world. I'm concerned that doing the file i/o in BACKEND1 is "expensive" - and I don't care about the performance of BACKEND2 as much.
A while ago, I wrote a web-based guestbook application that wrote it's own database.
My code was very amateurish, but, as it was my very first publication, I was very happy with it. Only about a month after I'd published it did I realize I'd made a huge mistake in the code.
I've only ever connected to a specific named instance of SQL Server, and it occurred to me that, if the SQL Server instance has a different name than the one I specified, it wouldn't work.
So, since my users will probably not know what the name of the instance of SQL Server that's running is, I thought adding a field where the user can specify it would help if they do, but what if they don't? My answer was to get the local instance, regardless of name.
I tried Data Source=.local;, Data Source=.; and other variants, but nothing worked.
If I have a very large function/sub in a class that is an instance method (i.e., not Shared), do I gain or lose anything by moving that to a shared method and then declaring a small stub method for the instance use?
I.e., I go from this: Public Sub MyBigMethod(ByVal Foobar As String) If String.IsNullOrWhitespace(Foobar) Then Throw New ArgumentNullException("Foobar") End If [Code] .....
My thinking is I save on memory size per each instance of the object. Because each instance only has to lug around the stub method which handles calling the shared version and passing an instance of itself to the shared method so that it can do whatever it needs to do. But I'll wager that I sacrifice a very teensy amount of speed because of the added function call overhead.
I am trying to copy a byte array into a Class that is in a third party library
Dim usr As New RSI_USER_RECORD Dim ba(RSI_USER_RECORD.RSI_LEN_USER_REC - 1) As Byte 'populate ba here usr = ba 'how can I do this? Is this even possible?
Here is the definition of the class (from Reflector)
I've got a VB2008 Form app and I'm trying to handle all my UI exceptions in a single method that handles the Application.ThreadException event. When I build a version and run it, this handler appears to be working fine. However, in debug mode, when an exception occurs the debuggernow halts on the exception and won't progress even if I hit F5. I want it to progress onwards and let the handler do its work. That's what it used to do, though I realize the difference is now there isn't a Catch block anymore. Any way to make the debugger keep going? It's making me nervous thinking perhaps things aren't set up right.
I am building an desktop app. In which I have to use a web browser. (with url: facebook.com) but I have a problem. This website runs in regular browser smoothly but in application it halts the computer and on and off I think it is due to heavy use of memory.
I have a project now where I will have a large chunk of CSV text, like 60 lines or more, and need to pass that from the client to the server. I know it would be weird with GET, but how would I store the multiline text so that I could later split it up, etc at my application?
I am developing a vb.net desktop application that reads mails from the mailserver using pop and imap. The application is almost fnished .The problem is the some time when a command is send to retrieve a mail using GetMessage() functions the applications is going to a halt. So I tried to run the the code in debug mode and ran line by line.
What I noticed is in the while loop in GetMessage() function is While Not tmpString.StartsWith(sPrefix) tmpString = reader.ReadLine msg = msg & tmpString & vbCrLf End While
The reader reads line by line at certain point of time it got hangs. Some times it ill catch an exception and give an error. I am pasting the exception below: "Received an unexpected EOF or 0 bytes from the transport stream". But some times the applications will hang and go for a halt. This applications has to run 24/7. So if it hangs in the night or it will be problem. What I want is if the reader is not able to read from the stream after certain time period. The control has to be removed from the reader and the function has to return a message. I am new to this the network level applications and I am running shot of time.
Public Sub connect() Dim objTCP As New TcpClient Dim sslstream As Net.Security.SslStream Dim reader As StreamReader = Nothing [Code] .....
I have created an application using VB 2008 Express on an XP machine. I created a deployment package and installed it on other XP machines with no problems. I tried it on a Vista machine and all seemed well until performing a serial port function. It simply halts/locks up. I have to close it in task mangler. I thought I would circumvent the problem by building and publishing it on the Vista machine. While running the program from VB express the program functions perfectly on that Vista machine, but when I publish and install it on same it still crashes at the serial portion of the program.
Is is possible to import or migrate .txt files to my database? Im using VB 2010 Express. Im just a newbie in Visual Basic and I need to import a text file into my system.
I have a Client/ server application where the Server is in java and Client is in Vb.net.When i send large string from client to server am not receiving complete text.code attached below.
client-- VB.net- Try Dim clientSocket As New System.Net.Sockets.TcpClient()
