File I/O And Registry :: Reading Files From A Large Directory?
Jul 15, 2009
I wrote a cleanup program to go through some directories and delete files based on if the creation date is older than say 6 months. It works fine with some of the directories I have that contain around a few thousand small files. However, there is one directory (that contains small backup files from another program) that is loaded with over 300,000 files and it locks up on me as soon as I read in the first file in that directory.I am convinced it is the directory has too many files in it to open it. The server that the directory is on is slow. It takes a half hour to open the directory while on the server itself. I know it will take forever to delete the amount of files I want to delete, but I don't understand why it gets stuck and hangs there with no error message.Here is where I get stuck. Listbox1 is the directory I'm attempting to access
For Each selectFile In My.Computer.FileSystem.GetFiles(ListBox1.Items.Item(Count), FileIO.SearchOption.SearchTopLevelOnly, "*.*")
compFile = Path.GetFileName(selectFile)
[code].....
View 6 Replies
ADVERTISEMENT
Dec 11, 2011
I wrote a program to download and play AVI files the files are large in size and long time wise. I have the files playing as they are being downloaded but i can't see how long the file is or seek reliably. i was reading that the AVI headers are in the last 512kb of the AVI file *why would't they put it at the start* lol so my question is is there a way to download the last 512Kb of the file i'm downloading. i have the Bytes Read and Length of the file while its downlaoding i'm just not sure where to go from there, or how to do it atleast. If i had the length in time of the Avi file i could set the trackbar to be able to seek properly. or if someone has an idea how i could get the time of the video by using fps and some math i could prob do it that way too but idk how i would tell how i can find how many kb are in the fps i'm sure it changes so think that way is't going to be reliable. how to get the last 512kb would be the best option not sure if it can be done even.
View 4 Replies
Aug 28, 2009
I am using zipforge.net to archive files in a small backup program I am creating. Everything works fine however if I try to archive a large number of files the process of zipping them gets slower the further along in the job it gets.
Example: I have a directory that contains a little over 100,000 files in it. Each file is only about 200k of text. When I use the zipforge zip class it starts zip'ing very quickly but as it gets about 40% of the way through the directory it starts to slow down... at about 60% its working at a snails pace.
Memory usage for my solution also goes up proportionate to the archive file size as it grows. Currently it takes around 26hours to backup this directory which only contains 4 gigs worth of information.
If I use a program like winzip I can zip the folder in less than an hour. I am thinking there has to be a way to write to the end of the zip file without having to open it every time or without keeping it open. My backup software scans the specified folder and writes each file path and name into a text file. Then it reads the text file line by line and passes the path + file info to the zipforge addfile().
View 1 Replies
Sep 1, 2008
Alright this is what im working with. Im Working with vb.net 2003.I have (X) amount txt files in "C:Marble".In each txt files there are 3 lines.I want to read the first line of each txt file and display them in a listbox. [code] Things i need to Find:
-1 How to tell how many files are in a folder
-2 How to create a loop were it reads the first line of each file in that folder
View 13 Replies
Apr 24, 2009
An application that my company uses stores report files with a .dat extension for use with their software. These files can be opened in notepad and the basic report information can be seen as text. I am attempting to write an app that opens and reads data from these .dat files and displays it in a checked listbox. My hangup thus far is that I can't figure out how to read the data from the .dat files since they are not .txt files. Can anyone point me in the right direction?
[Code]...
View 5 Replies
Aug 23, 2011
I'm trying to return a integer / count of the ammount of files in the directory and sub directories in the 'Windows' folder, I need to be able to do this as fast as possible (no lag) and tell when it's done, hope I explained this right.
View 3 Replies
Aug 14, 2008
I require some help in saving and reading files in Line by Line format, How would I save to a specific line in the text file, or read a specific line?
[Code]...
View 4 Replies
May 8, 2010
I am trying to find a way to allow my users to choose the directory to save files into, THe file name is automatically generated as it has a naming convention and i am able to pre program a location to save to but i d like to be able to allow my users to decide themselves which directory they want to save the files to and to have the ability to choose which directory to save to
[Code]...
View 2 Replies
Sep 17, 2010
Which is the most performant way to read a large csv file in .NET? Using FileStream? or another class
View 4 Replies
Sep 12, 2010
I am writing a small tool that identifies digital scielence in non compressed PCM WAV files. Its purpose will be to scan files approximately 1 hour long, containing mostly scilence and spltting the periods of non scilence into files and discarding any digital silence which is encoded as null bytes.I am wondering what the most efficient way of scanning the files is. I am able to derive the number of bytes for each second from the file header (in my case 16000) so am considering reading into a buffer of this size then scanning byte for byte, if I identify a non-null byte I will start siphoning the data off to another file.My issue is with scanning large expanses of digital silence which could be many megabytes of data which would require examining each byte to determine wether or not its digital silence. Surely there is a more efficient way of skipping them as I am not interested in them.
View 7 Replies
Jan 30, 2012
My problem is I have very large text files (approx 2GBs+).They have records in them based in one per line.Each line is not the same length and the data can be different lengths all the time.I am currently reading the file line by line, then splitting the data by common characters in the records. To process the full file it currently takes 3hours. This is way too slow for its purpose.
View 7 Replies
Feb 10, 2010
I'm tryng to read an excel sheet that has more than 255 cell. I'm using the following code:
Dim MyConnection As System.Data.OleDb.OleDbConnection
Dim MyDataSet As New System.Data.DataSet
Dim connectionString As String = ConfigurationManager.ConnectionStrings("PCOConnectionString2").ConnectionString
[Code].....
this code throws me an error that indicates me that there are too many rows to read. I found, seaching through forums, that it can't be more that 255 cells on the sheet i'm reading.
How can i read this sheet that has over 500 cells?
View 5 Replies
Nov 23, 2011
I'm busy with an applicaton which reads space delimited log files ranging from 5mb to 1gb+ in size, then stores this information to a MySQL database for later use when printing reports based upon the information contained in the files. The methods I've tried / found work but are very slow.
or is there a better way to handle very large text files?
I've tried using textfieldparser as follows:
Using parser As New TextFieldParser("C:logfiles estfile.txt")
parser.TextFieldType = FieldType.Delimited
parser.CommentTokens = New String() {"#"}
[Code].....
View 2 Replies
Aug 27, 2008
I am writing an application that connects to a specific com port. This com port can change depending on how many physical ports are on the PC, and if other USB devices have installed com ports before this one.
I have the communication to the com port all working great, but I need to get the com port description not just the "COM#" which is all I currently get.
I found in the registry where the description is stored but am having trouble doing what I need.
In this key:
Code:
HKEY_LOCAL_MACHINESYSTEMControlSet001ControlClass{4D36E978-E325-11CE-BFC1-08002BE10318}
there is a varied number of sub keys. The are numbered 0000-NNNN depending on how many ports you have. Inside each of these keys, there are multiple values, although I am only interested in one or 2 of them.
My problem is, that I do not know how to get a list of the subkeys (the 0000-NNNN keys/folders) from the original key I posted. I have looked through a few registry tutorials, and I can view the default value in the listed key, but I could not find how to list the subkeys of the listed key.
Does anyone have any sample code they could share that shows how I can get a listing of subkeys, and then run through those subkeys checking for a specific value in one of the strings contained in the subkey?
I am using VB.net 2008 and Windows XP.
View 1 Replies
Apr 20, 2010
i want to read a set of files from a particular directory( for example: D:Date has set of .xls files like D1.xls, D2.xls, D3.xls). if the the file is not there then it should throw error.i am not getting how to to do this VB.Net?
View 1 Replies
Feb 13, 2009
I know I've asked this before and for some reason I can never remember how to do it. I need to loop thru a directory structure and for every *.txt file, I need to open it and read it. This is a little application that basically searches for a given string. How do I do this?
View 2 Replies
Jan 30, 2012
My problem is I have very large text files (approx 2GBs+).They have records in them based in one per line.Each line is not the same length and the data can be different lengths all the time.I am currently reading the file line by line, then splitting the data by common characters in the records. To process the full file it currently takes 3hours. This is way too slow for its purpose.
View 4 Replies
Jul 6, 2011
I've a problem reading text file using StreamReader. The file have between 500 000 and 1 000 000 lines.When I try to read it in a cycle, I get an error. That's why I've tried the StreamReader.ReadToEnd method. It worked fine. I've get the entire contents of the file in one string. So far everything is okay, but I've a small problem searching this huge string. I have to reformat the string to my desired format. I'll try to be more specific: The format of the input file is as follows:
50471100 8 2 6 5 0<LF><CR>
00000016 365442 12231<LF><CR>
00000026 112166 31133<LF><CR>
<end>
[Code]...
View 7 Replies
Mar 24, 2009
I have created a project in VS2003.Net under VisualC++.-->Windows32 Application. I have to make an executable file that I will be able to run from different machine.The purpose of this file is to read some files that are in a particular directory. I will run the VSC++ .exe file from this folder. So I have to use some code to get the currentdirectory(directory where the current exe file is existing). After getting the directory, I have to find all the files in this directory that has an extension of .config.
View 2 Replies
Feb 1, 2009
The following code suppose to:
1. Read line-by-line a txt file with more than 500,000 lines, (each line 521 characters long)
2. extract an ID No from the line
3. query from a database for LCCIStatus
4. concatenate the value of LCCIStatus to the line
5. write the line to sample.txt
My problem is, this code works perfectly with the test file of 8000 lines but fail with the actual files which have over 500,000 lines. FYI, the test file contains data which I cut and paste from the actual file.
[Code]...
View 5 Replies
Aug 26, 2011
this code takes about 30 mins and high cpu usage, what is the problem
Do
strLine = objReader.ReadLine()
If strLine Is Nothing Then
[code].....
View 3 Replies
Mar 3, 2009
i need to read a large file stream binary my code is Dim sr As New IO.FileStream(srcFile,
[Code]..
View 10 Replies
Aug 26, 2011
this code takes about 60 mins and high cpu usage for a text file of 90,000 lines, what is the problem..[code]
View 9 Replies
May 4, 2009
I need to write data to a file, preferably in binary format, but I am unaccustomed with the concept. Where's the easiest place to get the basics? I could come here with a specific need, but I'm at the point right now where I am more willing to work within the confines of keeping it simple.
Here's what I know:
1. how to open a new file
2. how to specify the record length
3. how to close the file
Some specific questions:
Does the record length have to be constant throughout the file?
Can I read the nth record without reading the whole file?
View 8 Replies
Oct 22, 2009
I want to search a directory and get all the names of the "Folders" within that directory and place them into a string array. Note: No Files just folders
View 3 Replies
May 4, 2011
I'm trying to read a binary file from a FileMaker 11 container field using Filemaker's own ODBC driver. I was able to write files to the database and this works fine. retrieving them manually works fine and the files look OK and are not corupted. However when retreiving them using VB.NET, and if the file size is approx > 5MB, I get the following "uncatchable" error (yes thats right, I cant "Try Catch End Try", it just crashes):
System.AccessViolationException was unhandled
Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
[code].....
View 1 Replies
Sep 24, 2008
It keeps throwing an exception like this "Cannot create a file when that file already exists" at this line:
Code:
Directory.Move(contentDirectory, archiveDirectory)
I have checked, and the directory that I am moving, does exist at the source, but does not exist at the destination. I have tried including some console.writelines to make sure my variables are correct, and they look good. The first bit of code works where it creates the new directory, but fails when I try to move anything over to that directory.
Code:
Sub Main()
Dim Guid As String = "test"
Dim archiveName As String = "scripttestone"
Dim contentDirectory As String = "C:content" & Guid
Dim archiveDirectory As String = "C:archive" & archiveName
Dim TodaysDate As Date = Now()
[Code] .....
View 5 Replies
Aug 9, 2009
I am trying to read all the attributes, there are 3, from a single node. This is what I tried from an example I found:
Code:
Dim xl As New XmlDocument
xl.Load(XMLfile)
Dim rn As XmlNode = xl.SelectSingleNode("p1")
[code]....
I have used a messagebox to show the xl.OuterXml, I see the file is loaded.
View 3 Replies
Nov 5, 2010
friend of mine has no HTML knowledge so I'm attempting to write a program that replaces certain parts of a html file to suit his needs.I've edited the HTML file and marked certain parts with "tags" like this:
#IMAGEURL1# I have a textbox where he can copy + paste an image URL and hopefully the #IMAGEURL1# is replaced with the contents of the textbox.
So can someone please enlighten me to-as how to open a HTML file (there's no textbox to display the contents just yet, I'll add one if needed), find the specified text and then replace it with whats in a textbox.
View 1 Replies
Jun 14, 2011
I am trying to import a TAB (NOt comma) delimited text file into a DataGridView. The following code works fine if I have a comma separated file. All I have to do is change the FMT to "Delimited".It just does not work with FMT=TabDelimited. All columns are read into single datatable column. The text file is ANSI text and I have double checked to make sure Tabs are tabs and not spaces, even exported a sample Tab Delimited file from Excel.Can this even be done using Text Driver? [code]
View 2 Replies
