I have a large text file which I want to split into many different items.
On the next file I have a time between each item it's like this
01:20 a.m.
01:44 p.m.
In between these items is information like you would see in a log file.
How can I split these items like this?
I have a large file (2.7GB). I need to split it into smaller files. How to split a large file into smaller files using VB.NET 2003? Cannot use LINQ and the resources (cpu and memory) on operating environment are very limited (it is a shared hosting environment).
View 1 RepliesThe exe can call functions and forms in the dll's that are part of the solution BUT is there anyway
for the dll's to access forms, dataset's etc in the main exe project?
It doesnt appear that it works that way.
I forgot to add this is a VS2005 application.
I have a large financial application which I am migrating to VB.NETIt has two main components: charting and financial statements.I want to know if it is possible (and how) to create a main exe, and then two separate projects for the two components (as dll's?).
View 3 RepliesI'm really hoping I can describe this question in an understandable way. This is a puzzle that I have not been able to begin to solve even though I (mostly) understand it. I'm just not sure where to start, and I'm really hoping someone out there can get me headed in the right direction.
I have a LARGE table of data. It describes relationships between objects. Let's say the Y-axis has items numbered 1-1000, and the X-axis has items 1-1000 also. If item #234 on the Y-axis is related to item #791 on X, there will be a mark in the table where the row and column cross. In some industries this is referred to an a Truth Table. One can, at a glance, see how many items in a system relate to each other. The marks in the table can help to identify trends and patterns.Here's some other helpful stuff about the nature of the table:
The full range of the number of relationships (r) for each item on either axis can be 1 <= r <= axisTotal.The X and Y axis will share common items, but each axis will also have items that the other axis does not.Each item will only exist once per axis. It can be on X and Y, but it would only be on each one 1 time.The total number of items on each axis will most likely NOT be equal. Each axis could have from 50 to 1000's of items.
The end result is that this is going to be a report that needs to be printed. We have successfully printed a table that had about 100-150 items on each axis on an 11in X 17in piece of paper. Any more than that and it begins to be so small it's unreadable.
What I am trying to do is split the super large tables into smaller tables, but related points need to stay together. If I grab item 1-100 on X then I would need each item they relate to from Y.I've generated a number of these tables and, while the number of relationships CAN be arbitrary, I have never seen an item relate to all other items. So in real practice the range is more like 1 <= r <= (10% * axisTotal). If an item's relationships exceed this range, it can be split up into multiple tables, but that is not optimal at all.
At the end of the day I think we, and our clients, would be happy if a 1000x1000 item table was split into 8 to 10 printed pages of smaller, related tables.One other thing worth noting, there will be no empty rows or columns in the table. Every item on both the x and y axis will relate to at least 1 item on the opposite axis.
I have one large XML file from one of our vendor and I am trying to pars it using Linq but it looks some wrong logic I am using.
This is xml file:
HTML
<Psw xmlns="http://localhost">
<exid>20</exid>
<Mes><Me>
<doc><ps>
<ghder> .....
The code I am trying:
Dim doc As XDocument
doc = XDocument.Load(TextBox1.Text)
Dim qList = (From xe In doc.Descendants.Elements("ghder") _
Select New With { _
.mid = xe.Element("MId").Value, _
.cdate = xe.Element("cD").Value _
' As well I need to get the value for pn child attribute }).FirstOrDefault
MsgBox(qList.mid)
Error for DBNull ' Object reference not set to an instance of an object.
I'm wanting to create a .chm file for a large project I've just completed.
Is there a easy way to create one of these? one that has Contents, Index, Search and Favourites section.
The following code suppose to:
1. Read line-by-line a txt file with more than 500,000 lines, (each line 521 characters long)
2. extract an ID No from the line
3. query from a database for LCCIStatus
4. concatenate the value of LCCIStatus to the line
5. write the line to sample.txt
My problem is, this code works perfectly with the test file of 8000 lines but fail with the actual files which have over 500,000 lines. FYI, the test file contains data which I cut and paste from the actual file.
[Code]...
I am trying to parse a very large text file for certain strings. The text file is part of a level-making software for an old game I play. The text file basically contains all the information the level designer software needs, but the only important bit is the 'texture information'. Basically what I'm trying to create is a little program that parses the text files and shows the user a list of every texture in that text file. The problem is, the strings denoting textures are not really easy to find, and I can't think of any sensible and fast way to get them...
[Code]...
I need to load a large txt file that is in a fixed width format. There are over 45K lines, so speed is important.I need to load one of the fields into a dropdown box and have another field (label) display the text of another field in the related line.I could import the file to an access db if needed, but would rather not as i also want the txt file to update from a link on a regular bases. So having it in a DB would be more work to process that part.[code]
View 1 RepliesI want to write a program to do Markov chain, but my states are quite large. First of all I calculate all the transition probabilities and revenues for all states(1381860 total states), and store in a multidimensional array. Public RevArr(0 To 9, 0 To 750, 0 To 282) As Long
After that the iteration of markov chain should use these as inputs to calculate the steady-state probabilities. But when I try to run the main code I got this error.Exception of type 'System.OutOfMemoryException' was thrown.
The following is the declaration of second array I add just another dimension for storing all the iterations, but I get this error. Dim stateprob(IT + 1, 0 To 9, 0 To 750, 0 To 282) As single
I have listbox which is populated from a text file with all the items fromt he 1st split, the text file looks like this:
Quote:
test l1c1 | test l1c2 | test l1c3
test l2c1 | test l2c2 | test l2c3
[code].....
this code takes about 30 mins and high cpu usage, what is the problem
Do
strLine = objReader.ReadLine()
If strLine Is Nothing Then
[code].....
I have a File i need to read and extract a few values from each line in the file. my problem as it stand the file i'm reading has some unconsistatcys as all of the Strings are seprated by " " but the integers are not they are Without any Speech marks to define them. I have thought of a methord to deal with this but i can't for the life of me think how to achive this.
As the file has unconstancys to be used with a standard " " extractor, the only consistancy is that there is allways a " before the integer and after attached to another column. this could be used i thought.
a Sample of the data is only one line:
[Code]...
this code takes about 60 mins and high cpu usage for a text file of 90,000 lines, what is the problem..[code]
View 9 RepliesI am trying to split a long string based on an array of words. For Example:Words: trying, long, array Sentence: "I am trying to split a long string based on an array of words."Resulting string array:Multiple instances of the same word is likely, so having two instances of trying cause a split, or of array, will probably happen.
View 5 RepliesThe file is a UTF8 text files.
Each characters has varying number of bytes and each line has varying number of characters.
Does vb.net has table of line numbers to byte location function or something like that?
Also after that how to read that?
I am writing an application in VS2010 using VB.net.The application is relatively simple and I would expect the .exe to be less than 1 Meg.I have written a couple of applications that are quite a bit larger that are less than 2 Meg.I am compiling in "Release" mode.The file size is 29,127 KB (in debug mode it is just 29,167 KB)Where do I start looking to find why the .exe is so large?
View 2 RepliesI have problem, how I can import HEX file to my program in VB .NET? And I have question if I can import, What size of file I can import? I have 200MB - 400MB HEX files?? Can I import so huge files?
View 18 RepliesI am trying to read this XML document.
An excerpt:
<datafile xmlns:xs="[URL]" xmlns:xsi="[URL]" xsi:noNamespaceSchemaLocation="wiitdb.xsd">
<WiiTDB version="20100217113738" games="2368"/>
<game name=" Wanted: 50 Wacky Jobs (DEMO) (USA) (EN)">
<id>DHKE18</id><type/>
<region>NTSC-U</region>
[Code] .....
It just skips the "While iter.MoveNext" part of the code. I tries it with a simple XML file, and it works fine.
I have a simple app that reads from a very large text file, and returns a value if a string is found. I can instruct users where to download the file, and where to put it, but it would be nice if I could embed the file with the publish, so that the program knew where to look by default.Getting users to download a seprate file is painful. This file has 1.4 milion lines of text. I really need it to look for the file in a predictable place and be able to run against that by default for most users. I can have experienced users browse for a new file, but most people aren't into that much thought.
View 8 RepliesI'm trying to load a large CSV file into a Microsoft SQL Server Compact 3.5 database. I've tried using the following: Using MyReader As New Microsoft.VisualBasic.FileIO.TextFieldParser("filename.txt") MyReader.TextFieldType = FileIO.FieldType.Delimited MyReader.SetDelimiters(",")Then splitting the data with MyReader.ReadFields() etc, before using this data to add rows to a dataset in a database table in my project. However, my CSV files are very large, at above 9.5 million rows, and this takes forever, if the computer doesn't crash also. Does anyone have a better idea for what to do? I would like the CSV file to be loaded into the database table, to enable me to sort it, and undertake some querys and maths. The CSV data structure is:2,193,761.40000000000012,43,1510.22,8,1929.60000000000012,22,2564.52,22,2791.70000000000032,19,2971.6000000000004
View 3 RepliesI need to read the second-to-last line of a very large log file.
I can't read the entire thing into memory, count lines, etc etc. I can't use Filestream.Setlength because that needs readwrite access and the log will be opened by another application. And it has to be fast. However, the line ends in a cr/lf meaning the last line is actually empty. Been struggling with this all day, and its hurting my head! Not good on a Friday!
I have a function that can read the last line but can't get it to go one line up. I can get it to read characters from the end of the line, but that's not much of a help with a variable length line!
Maybe fs SeekOrigin would work as it could run backwards looking at an example from MSDN - need to get the data before the last cr/lf and end at the next one... hmmm... problem is that that example also writes the text backwards as well.
I'm monitoring the log for particular entries for issues that are causing us grief at the moment.
My problem is I have very large text files (approx 2GBs+).They have records in them based in one per line.Each line is not the same length and the data can be different lengths all the time.I am currently reading the file line by line, then splitting the data by common characters in the records. To process the full file it currently takes 3hours. This is way too slow for its purpose.
View 4 RepliesI've a problem reading text file using StreamReader. The file have between 500 000 and 1 000 000 lines.When I try to read it in a cycle, I get an error. That's why I've tried the StreamReader.ReadToEnd method. It worked fine. I've get the entire contents of the file in one string. So far everything is okay, but I've a small problem searching this huge string. I have to reformat the string to my desired format. I'll try to be more specific: The format of the input file is as follows:
50471100 8 2 6 5 0<LF><CR>
00000016 365442 12231<LF><CR>
00000026 112166 31133<LF><CR>
<end>
[Code]...
Is there anyway to reliably know when a large file has completely finished copying to a particular folder? For example, Computer1 copies a large file to Server1Share1. On Server1Share1, I want to do something AFTER the file is done copying, without Computer1 intervention.
View 2 RepliesI have a 133 Mb file which contains almost a million records. Currently, a user loads this file into KEdit (a great editor for working with large files) and changes occurrences of a dollar sign $ to blank and takes negative numbers represented as such with parentheses and changes that to a negative sign. That is, (5000.23) would become -5000.23. So the leading ( becomes a - and the trailing ) becomes a blank. I believe the only occurrences of ( and ) are around the numbers I want to change, so I don't have to worry about changing something that should have been left alone. Using VB.NET in Visual Studio 2008, is there a "painless" way to do this other than reading the file one record at a time and searching/replacing and writing the record out? While not really painful, I am worried about how long that will take (to run, not to code ). Is that a valid concern?
My long range goal is to automate many of the routine file preparation tasks my users do. We get an input file from 15 clients. The file can be in any format the client has chosen to give us, and it is our burden to reformat any errant fields into an acceptable format for insertion into our database.
Is there a quick way of finding a character?? I have this error
Invalid character in the given encoding. Line 1, position 7173.
Eak, I need to know what this is but don't really want to count across 7000+ letters
I have a large wikipedia dump that I want to cut into different files (1 file for each article). I wrote a VB App to do it for me, but it was quite slow and crapped out after a few hours of cutting. Im currently splitting the file into smaller 50mb chunks using another app but thats taking a long time (20-30 minutes for each chunk). I should be able to cut each of these up individually if I do this.
Does anyone have any suggestions of a way to cut this file up quicker?
I have created a simple Backgroundworker process to copy a large file (30GB). Is there any way to report the progress of that file copy?
I'm using System.IO.File.Copy to perform the copy. I've seen a few posts/blogs that suggest comparing the bytes copied with the size of the source file but that seems like a huge overhead in this case.
