Searching Large Binary Files And Data Carving
Mar 12, 2011
I am trying to develop an application that will let me do the following
1) Be able to search to large binary files for a specific byte array (header) and to continue searching for another byte array (footer). Then i want it to copy the section of the binary file between these 2 locations to a specified file. Then the program is to continue searching for any other header/footers An example would be search a large file and carving out the jpgs it finds etc. This routine would have to be able to handle raw images of 20gb or more if needed.
View 10 Replies
ADVERTISEMENT
Jan 27, 2011
I have some functions written in C that read & write large binary files (> 4 GB) using routines CreateFile, ReadFile, WriteFile & SetFilePointer, that use the HANDLE data type to access the file. Some others who are more familiar with Visual Basic would would like to be able to call my C routines to work with such large binary files. We particularly need SetFilePointer to jump around with the files.
[Code]...
View 2 Replies
Apr 29, 2010
How would I use the BinaryWriter to append binary data to a file that is very large? Say a file that is 3GB in size. The BinaryWriter's write method will only accept an integer for the index value. I've attached a screen shot of what I'm talking about. If I were to enter a value that exceeds the size of an integer on my 32bit Windows XP system, which I believe is 2147483647 it will reject it right?
View 4 Replies
Dec 2, 2010
I am creating a software to carve data from an image file( removing data which can be deleted and undeleted) I am using the header and footer method. The problem is that am not able to save the data in between the header and footer. The data i want to save is raw data , in hexadecimal.So after getting the data, i would be able to get it back to its normal form.
View 1 Replies
Sep 21, 2010
I am working on a program for a customer and am working in VB2010 in VB language. I have two rather large files that contain data. File #1 is over 300,000 lines of data. File #2 is over 100,000 lines. The files are CSV text files. On a User Form, there are two values, "StartTime" and "StopTime". This is the process I am doing:
1. Read both files into ArrayLists
2. Get item #1 of ArrayList #1 (we'll call this "A")
3. Get item #1 of ArrayList #2 (we'll call this "B")
[code]....
View 6 Replies
Feb 28, 2012
Some of my users are collecting data files this size and wish to load it in to my tool. Is there a way to load large files of string data w/o getting a OutOfMemoryException? I have tried many different storage methods (Array, List, etc) but always without success.Before anyone asks 'why load it all at once', let me say the nature of my app is speed critical and it must be so.
View 3 Replies
Apr 14, 2009
I search but I didn't find the answer all of them explain how I use binary files only
View 2 Replies
Nov 2, 2009
What do you think what is faster:
1. writing 1000 records in a binary file on a server? or
2. passing the same 1000 records in a database on a server?
and
1. search a specific record in a binary file? or
2. search the same specific record in a database file?
View 2 Replies
Jun 27, 2009
I haven't programmed for some time, so I'm a little rusty Basically I'm storing HTML within a variable, and I need to count the amount of times a specific element appears e.g. <h3 class=r> for now.
I thought of using a for each loop, and passing the string above into the parentheses of the loop (using a variable ofc) however for many reason's, I don't think this has a chance of working (won't bother explaining why, as I assume you can guess). Then thought of using the .Contains method of the string class, but I simply can't think of how to use it with what I'm trying to do.
View 2 Replies
Mar 27, 2012
I have a requirement to search large piles of text for keywords
The "keywords" are known in advance - but the number is variable.
The text is large - I've had 50000 byte strings in tests already...
View 22 Replies
Jul 12, 2011
My app is a Bible Reader. At load up I can choose Genesis Ch1 and its entirety pops into a rtb. I enter a word to search for like 'God' and hit the SEARCH button. The results I get are initially what I want. I get the verse 1:1 In the beginning God created the heaven and the earth. But then the app replaces the script with other words containing God, but all jumbled up. Here is the code I am using: Private Sub ToolStripButton2Search_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles ToolStripButton2Search.Click Dim stSearch As String stSearch = Me.ToolStripButton2Search.Text For Each Line As String In File.ReadAllLines(Path.Combine(Application.StartupPath, "Bible VersionsKJV 1611King James Version of the Holy Bible.txt")) If Line.Contains(Me.ToolStripTextBox1.Text) Then Me.RichTextBoxViewer.Text = (Line) End If Next LineBryn Ryver
View 2 Replies
Dec 25, 2009
Couldn't think of a better title.(Background on the problem/me)Okay, so, first question/post here, so hi. Now that that's done with, the information pertinent to my problem. I'm fairly new to VB (and programming as well, aside from screwing around with C++ and learning assembler(well, attempting is the better word) god knows how many years ago), and have only seriously been programming for about under half a year, and my skill level is about at that stage. Only been using VB.Net, nothing older. Depending on the time of day and if I'm home or at school, I fluxuate between VB express and Vis Studio 08. Umm, this program I'm having trouble with was on a test that I took yesterday (took the problem home with me cause I really wanted to figure out what was wrong with it).
The stipulations of the test were:
No For->Each Loops
No using Built-In Sorting or Searching Functions
[code].....
View 4 Replies
Feb 23, 2011
I have been googling this and have not come along a working solution for an entire day. I don't know anything about binary data types (as I've never knowingly used them) and I'm trying to write a binary value that I see in the registry, to the registry. Now, all I see is the following [shown below]... and if I try to pass that as a string to the RegSetValueEx in the WinAPI and of course it errors out...
I do not know what 'numbers' I need to pass into the lpData As Any, argument of RegSetValueEx (i tried a bit array) in order for it to come out as the following [shown below] in the regedit. I really have no idea, and my tests to place random numbers in the bit array just produce corresponding random "figures" as visible in regedit that I do not understand how to 'tie' them together logically. here is the culprit!
[Code]...
View 2 Replies
Dec 11, 2011
I wrote a program to download and play AVI files the files are large in size and long time wise. I have the files playing as they are being downloaded but i can't see how long the file is or seek reliably. i was reading that the AVI headers are in the last 512kb of the AVI file *why would't they put it at the start* lol so my question is is there a way to download the last 512Kb of the file i'm downloading. i have the Bytes Read and Length of the file while its downlaoding i'm just not sure where to go from there, or how to do it atleast. If i had the length in time of the Avi file i could set the trackbar to be able to seek properly. or if someone has an idea how i could get the time of the video by using fps and some math i could prob do it that way too but idk how i would tell how i can find how many kb are in the fps i'm sure it changes so think that way is't going to be reliable. how to get the last 512kb would be the best option not sure if it can be done even.
View 4 Replies
Jan 29, 2009
How do you recommend I copy large files over the LAN (>20GB)Currently I am using File.Copy.Another question I have is how can I implement this class to be self contained, meaning that it would report progress to the object that called it? or do I have to define a delegate in the calling object?
Code:
Public Class cCopier
Private Delegate Function CopyProgressRoutine(ByVal totalFileSize As Int64, ByVal totalBytesTransferred As Int64, ByVal streamSize As Int64, ByVal streamBytesTransferred As Int64, ByVal dwStreamNumber As Int32, ByVal dwCallbackReason As Int32, ByVal hSourceFile As Int32, ByVal hDestinationFile As Int32, ByVal lpData As Int32) As Int32
[code].....
View 1 Replies
Jul 24, 2010
The underlying connection was closed: An unexpected error occurred on a receive.If I look on te FTP server I can see that the file has been uploaded entirely which led me to believe that the control channel was timing out. I have tried modifying the timeout on the FTP server as well as the ReadWriteTimeout in my code to no avail.Here is my code...
Dim ftprequest As System.Net.FtpWebRequest = DirectCast(System.Net.WebRequest.Create(destination), System.Net.FtpWebRequest)
ftprequest.Credentials = New System.Net.NetworkCredential(username, password)
[code].....
View 1 Replies
May 13, 2009
I am opening a large PDF file using the following
Process.Start(Application.StartupPath & "PREVI.pdf")
I have two questions:
1) Is there a way to go directly to a page number?
2) Can I write a code to have the table of contents show up on the left side of the document so users can click on to go to different sections of the document, not sure if this needs to be done while creating the PDF or if we can do it by code.
View 4 Replies
Dec 20, 2011
If you were to upload a project to any site but are concerned about the large file size, what are the only files you need to zip up for your project to work for either VB 2008 or VB 2010? So far I know you don't need the exe's, the pdbs, and xml files.
View 5 Replies
Sep 17, 2010
Which is the most performant way to read a large csv file in .NET? Using FileStream? or another class
View 4 Replies
Apr 27, 2012
Does anyone know what the best way to split up large files in VB.NET? These files can be in excess of 10GB. I have found ways of doing it by googling all day! most of the solutions I have found almost work. But what I really want to know is what is the most efficient way to do this?
View 1 Replies
Mar 12, 2012
I have trying to upload some large files using FTP. Here is the code I am using below.
Private Sub UploadFTP()
' Open file.txt with the Using statement.
Using r As StreamReader = New StreamReader("list.txt")[code]....
I basically loop the above to upload all the files in a specific directory. This works fine with small text files,but when I try uploading a large 30+- mb file it gives me an error.
Quote: The underlying connection was closed: An unexpected error occurred on a receive.
On
cls_stream.Close()
I understand this is caused by .Net not communicating with the server properly? I tried searching all morning to only get confused by all the different methods.
View 6 Replies
Jan 13, 2012
I can search all the images with .jpg extension and pass it to the fancybox gallery , The issue is i just want the images of particular productid for instance if a product has 5 images , and they are saved as (productid_imagenumber.jpg) , therefore a product with productid 99 will be saved as 99_1.jpg , 99_2.jpg similarly 99_5.jpg,I can pass the productID but i cant find a away just to get the images of that productID , instead of getting all the images which is done by the function below:
Dim directory As DirectoryInfo = New System.IO.DirectoryInfo("C:Images")
Dim allImages() = directory.GetFiles("*.jpg", SearchOption.AllDirectories)
Dim strContent As String = ""
For Each image As FileInfo In allImages
[code].....
View 1 Replies
Jun 24, 2009
I want to create Desktop Search Engine using vb.net. While searching on internet someone suggested that I should use B+Tree indexing for quick results. But I don't know how to implement B+Tree indexing for searching file.
View 7 Replies
Jan 19, 2010
I'm creating an app that has check boxes that will define a search. The files all reside in one folder. I've looked at some tutorials that use io.directory.getfiles and I can get this to work. But what I think i want to use is: io.path.getfilename because I don't need the path (though I'm about ready to make an exception).
When should I use io.path.getfilename and can I use that method to do a search? I can't figure that out. I would think it would react as io.directory does but I am incorrect.....
Here is the code. Its short becasue I'm just figuring things out first:
Public Class Form1
Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
Dim afiles() As String
[Code]....
The commented out line is what I can't get to work and I'm thinking that is what I need to work.
View 1 Replies
Nov 29, 2011
im using the following lines to search for zip files in a folder i need to update this to allow searching for .zip, .txt, .evt and .rar files now
[Code]...
View 8 Replies
Jun 20, 2011
On a image file (PDF), the OCR has recognised the picture and text. However on Visual Basics, how do you search for a text on this image file? The primary goal is to allow a text search (i.e POxxxxx) on all the image files (PDFs). The returns of the search will be the assciated image files where the text (i.e POxxxxx) is found on.
View 1 Replies
Nov 27, 2009
I used backgroundworker to receive file segments from a server, it taking the segments and save them directly to a binary file. the problem is,I using a global variable to store the "FreeFile()" number.when "backgroundworker" receive "START" it do F = FreeFile()
when it receive "Segment", it should append to that file, but the problem is ... backgroundworker no more recognize the file (which its identifier saved in global variable F) and telling "Bad File"?!
whats the soultion?I think about opening the file and close it each time (not leave the connection to the file available). But I didn't figure out how to append to Binary files, It always replace their contents. and If I use "Append" option it corrupt the file!
View 10 Replies
Jan 10, 2010
i tried out this [code]i can create and write in the .bin file but how to read it then?
View 20 Replies
Oct 19, 2010
For some reason the last PDF file to be written to the "final document" is the only PDF File that is visible when the "final document" is open. However, all of the bytes are accounted for in the file size...so for example: file1 = 60 bytes and file2 = 49 bytes. The final document has 109 bytes but only contains file2 when viewed in the pdf viewer.
[code]...
View 2 Replies
Jan 29, 2010
I have created an vb.net windows application. In the application i have 2 xml files (e.g., Pre.xml & Post.xml) & the 2 files are same structure.
My requirement is compare the pre & post xml files and get the differences between xml files. Like what are all the newly added nodes, removed nodes and modified nodes as well.
I tried with XmlDiffPatch.Dll, it is working for small size xml files, but in my case xml file size should be upto 100+ mb.
View 4 Replies
