VS 2008 Extracting Information From Websites
Aug 16, 2009
I'm currently using a webbrowser control to extract information from multiple websites. At any given time, my application could be searching through 1-80 different webpages across 1-4 different websites. On my computer, it runs pretty quick. I could go through all 80 sites in just a few minutes. But when on another machine with fewer resouces, it runs awfully slow. Taking up to 15 minutes. Which kind of beats the purpose, but not entirely, of my application.
I'll begin by explaining my methods, and go from there. When utilizing the Webbrowser control, I use While statements and Application.DoEvents, instead of the Webbrowser controls's DocumentCompleted event. I'm wondering if this is one thing that could be slowing down the process. Though I'm not sure why it would be. Also, would utilizing separate threads help with this? I can't imagine it would boost performance. But I'm not entirely sure. I think it bogs down because the application uses so much memory when running. On idle, the application uses about 9.5k memory. When running, I've seen it go as high as 110k.
View 4 Replies
ADVERTISEMENT
May 10, 2009
I am trying to take specific information from a web page and then process that information so that it can be sent to a label or text box.Previously, I had wanted to do this by using Regular Expressions. I've looked around and it seems that using regular expressions to parse information isn't always the best way because website aren't always coded to standards.Regardless, learning regex isn't working out too well for me. So, I was wondering if there was another way to do this?I was thinking that I might be able to use the web browser control. It would be ideal to be able to see the page in the form, select the information, and then be able to display it.
View 8 Replies
Oct 2, 2009
The method I'm currently using to extract html and the parse is via a WebBrowser control. I'm grabbing a collection of tags, sorting through the ones I want, and then pulling the innertext.
Doing this on my development machine is kind of slow, but manageable. At max, I can go through 60 different web pages across 3 different sites. It takes about 5 mins on my machine.
However, this app is targeted towards machine that have a quarter of the technology that my computer has. So, it takes anywhere from 10-15 minutes. This is less than ideal.
Does anyone know of any other method that I could do that would take less resources and perform a lot quicker?
View 7 Replies
Feb 3, 2010
My university has a website that students can use to check stuff like their lesson schedules, latest grades, exam times, etc. Just general information about you and your study (after you have logged in of course).For interested, this is the website (not much use without a login obviously, but still): [URL]I have been thinking lately that it would be nice to have a desktop application that could give me the same information. However, I have absolutely no idea how that would work, and whether or not it is even possible. I mean, technically it should be possible for me to get the information (how else does the website get it) provided it is not secured (even when using the correct login details). I am sure the website communicates with a database somewhere. Does that mean I could theoretically also communicate with the database (probably: read from it only, not write) for example by executing queries?If that is possible, how do I figure out which queries to use for the different pieces of information? Obviously I don't know the details of the database so I can't just send a random query and expect it to work...
View 8 Replies
Nov 1, 2009
I'm trying to get the version of IE and Google Chrome from the registry. I've successfully been able to get IE, but it seems chrome is eluding me.The following code is what I'm trying:
vb.net
Public Sub ChromeVersion()
Dim SoftwareKey As String = "HKEY_CURRENT_USERSoftwareGoogleUpdateClients{8A69D345-D564-
[code].....
View 2 Replies
Oct 2, 2009
The method I'm currently using to extract html and the parse is via a WebBrowser control.I'm grabbing a collection of tags, sorting through the ones I want, and then pulling the innertext.Doing this on my development machine is kind of slow, but manageable. At max, I can go through 60 different web pages across 3 different sites. It takes about 5 mins on my machine.
However, this app is targeted towards machine that have a quarter of the technology that my computer has. So, it takes anywhere from 10-15 minutes. This is less than ideal.I've bee reading up on the documentation on HTTPWebRequest. However, form what I've seen I can't seem to find a way to grab the HTML and then parse it with a method from HTTPWebRequest.
Due to the way I'll be using it, I won't be able to use RegEx to parse the data and was hoping there was a method within the HTTPWebRequest property to do it.
View 2 Replies
Jul 4, 2011
I have the following text, it displays like this in a textbox, but is all stored in one string:
[Code].....
But I cant think of a way to do it, the only thing I can think is to try and grab the text on the line after each "+CMGR:"
View 2 Replies
Feb 19, 2011
I have a lot of information distributed among some tables in a database (and must likely it will all be represented into oop). I will need to create text files with different formats from the same data depending on the request. My idea is to create a design that will require no coding if such formats are to change of be added.I was thinking in using some sort of XML magic trick to accomplish this but I have no idea how to. I am also open to new ideas.
View 1 Replies
Nov 7, 2009
I was just wondering how to extract or parse any particual tags (whichever I specify) from webpages. I know how to extract text and links from webpages, but I tried to use the same method from the following code for div tags, title tags etcetera and it doesn't seem to work:
[Code]...
View 2 Replies
Feb 5, 2011
I'm trying to identify a specific string within a text file, and then extra a couple of bits of information that follow from that.
Example snippet from text file:
>Date 5.3: Tech Manual {30201}
Freya Plumes (30201) - 1
mu
These are made from brightly coloured. I want to be able to find where it says "Tech Manual", and then assign "Freya Plumes", "30201" and "1" to variables for manipulating and exporting to another file. There will be many instances from the same text file with similar information, so I'd need the extracted strings popped into an array for later referencing.
View 7 Replies
Feb 25, 2009
I want to extract a specific information from a webpage.For example:url...So can I make the software see what's between "<h1>Your IP address is<BR>", to "</h1>"?And doing all this by using the webbrowser form in Visual basic studio 2008.
View 1 Replies
Dec 20, 2010
I want to extract some information from a web site (Bloomberg). This side has a search textbox with a onKeyUp event that, as far as I can understand, uses AJAX to create a table. Is it possible to insert some text in the textbox and then get the table created with AJAX? Given the source code of the page with the tabled already AJAXed (If you understand what I want to say here), I can easily retrieve the table. I just need to get the source code with the table already on it.
View 7 Replies
Aug 18, 2010
I'd like to look into extracting EXIF tags/data from jpg images. I've looked around but can't seem to see any simple example. Anyone able to point me into the direction of pulling out the model/date values etc out of EXIF data?
View 2 Replies
Nov 13, 2011
I would like to be able to show all the websites visited on a machine.
View 1 Replies
Mar 8, 2011
i was wondering if there is any way to visit a blocked website using visual basic 2008. i want to use during school in programing class.. just to let you know my teacher said he will let us get by with it, if we could even make something like this.
View 8 Replies
Jan 7, 2009
I would like to be able to show all the websites visited on a machine.
View 3 Replies
Jan 6, 2012
I read some posts in the vb.net forum for how to modify your host file on your computer adding the domain website to block. this worked great. Then i went to a website such as vtunnel.com and was able to view the blocked website through it. So to still block it I need to add [url]... to the host site as well as any others?
View 4 Replies
Apr 14, 2010
I want webbrowser to visit multiple urls that I filled in List(of String). I use a for each loop to iterate over the List, but the webbrowser only show the last url ask.com. I want webbrowser to visit and display all the urls.
Dim UrlListArray As New List(Of String)
UrlListArray.Add("http://www.google.com")
UrlListArray.Add("http://www.yahoo.com")
[code]....
View 4 Replies
Mar 1, 2010
I have the following code which sets internet explorers proxy settings to what ever i want and if i set it to use a proxy ip and port i get a 'Page can not be displayed' error. If i reset it back to the bloxx server again which is: http://gc-bloxx:8080 Then it works fine. What i am asking is, can i get the app to connect through 'http://gc-bloxx:8080' and then set the proxy ip and port so the app can connect to the internet and then unblock the websites?
[Code]...
View 1 Replies
Apr 10, 2009
I'm trying to get my program to go to the memberlist.php of a phpBB2 forum and gather all the links to the members websites.
Example: [URL]
I want the program to go to that website and save all of the websites (the ones that have the "www" image) to a list inside the program. I have no idea how to do this, but that's not all, considering most forums have several pages of members (this one has 32 at the moment) it has to go through all of the pages and gather all the links from every page..
View 6 Replies
Sep 7, 2009
I'm using Visual Studio 2008 on an XP machine and am trying to develop a web app that can interact with websites and carry out automatic tasks.One problem I have is trying to recreate the web requests for automatic interaction.
For example:I use a webBrowser control to allow me to surf a page manually. Then, once I am at a specific location, I'd like the automatic interaction to take over.Part of the website requires a log-in and exchanges cookie data.When I try to read the cookie info from the webbrowser control to build the subsequent automatic requests, the cookie content is incomplete.Upon further investigation, it appears that the site sets one of the cookies to "http only". This stops the value being visible to the webbrowser control.Is there another control or method I could use to retrieve this info?
View 1 Replies
Apr 25, 2012
I have been working lately on a program who extracts URl Source codes!The program does work with most of URL but not for MEdiafire URL!When i check the source code from the web browser i can see there is some code missing;y tried diferrent types of Encoding.
Example:This is the final source code extracted from WEBBROWSER(Firefox,InternetExplorer,GoogleCrome)
--------------------------------------------------------------------------------------------
<div class="mf_lightbox_btns lb-footer" style="text-align: right;">
<a href="javascript:void(0);" class="secondary btn" onclick="$('body').removeClass('has-virus'); return false;">Dismiss Message</a>
<a href="http:www.bitdefender.com/mediafire/fix-it.html" target="_blank" class="alt btn">Get BitDefender</a>
[code]....
View 1 Replies
Oct 20, 2009
All I've been able to do is create the form. We have to get information off of a website and load certain information back into the textboxes on our form.
Assignment 6 - Text Parsing Many applications require you to extract information from a block of text. We will be using this heavily in the project. This project is designed to give you some practice extracting information out of text files. You will need to open up a file, search the file for some specific content, and then copy that content and place it on screen.
Go to Jobs.com and select a state that starts with the same letter as your last name (if there is no state that starts with the same letter, use the second letter, then the third letter, etc.). For example if my name was 'John Byway', since there was no B or Y, I would pick a state starting with W - either Washington, West Virginia, Wisconsin, or Wyoming.
The idea is you want to extract all the jobs information out of the page. We want to put the jobs names in a combo box, so the use can pick any of the jobs. Ideally, this would also show the information about each job. You don't need to do that. However, you do need to extract the information and put it on the screen (When you load the next job, the old job information will be lost, and you will be left with the information about the last job loaded)
Tasks
1. Go to the above address, view the source in the browser, copy and paste it into a text file. You do not need to access the Source of the web page within your application.(in IE go to View / Source; In FireFox go to View / Page Source)
3. Extract each job title and place the name in the Combo box.
4. Find and extract the following fields. Note, some jobs may not list all of these. In that case, get as many as are presented.
a. Date
b. Title of job
c. Company
d. Location
[code].....
Indicate on screen how many jobs you found. Note, you only need to look on the first page of jobs?
View 1 Replies
Dec 21, 2010
I have a program made in VB that on a button click extracts a exe file from its resources to the HDD then runs the program and deletes it after it closes. I first used the desktop to extract the file to, but seeing as I need to use this program on several computers the account names will be different so this is of no use. I've tried using C: but access is denied.
View 2 Replies
Jun 21, 2009
I've found all sorts of threads about extracting file icons, and I'm able to do that no problem, but I would like to also extract some other information from a file (like you can find in the properties window), such as the file description. Are there any commands that are geared specifically toward getting this information?
View 1 Replies
Oct 19, 2009
I am wondering if there is an option of extracting fielpath using OpenDialog control.
I am using following code..
strFilePathAndName = openFileDialog1.FileName
strFilename = IO.Path.GetFileName(strFilePathAndName)
[code].....
View 2 Replies
Jul 20, 2009
I have been looking everywhere to find a way to extract the main icon from for example c:/test/file.exe and put the icon in for example c:/test/fileicon.ico i did find something but the icon looked dreadful, it must of been like 16 colours.
anyway if someone has managed to do this or knows how to do it please help before i go insane.
also i will settle for knowing how to do it with reshacker.exe instead which i also cant get to work.
View 14 Replies
Jan 14, 2010
I'm extracting icons from running processes and then adding them to a ListView. Some processes don't have icons or I can't get access to them, so the icons are sometimes all mixed up. The following is how I'm extracting the icons:
[Code]...
View 4 Replies
Jul 25, 2010
I have a string in this format :
[Code]...
View 13 Replies
Jul 23, 2009
Basically, the way it works is by comparing the client's version to the server's version.If the client's version is lower then the server's version, it starts updating. Right now I -think- I have it already downloading the file correctly, but am not sure.The source I have it set to currently doesn't have anything there, so if I am doing the downloading wrong, please say so.Then, it extracts the zip that it downloads using SharpZipLib.After that, it deletes the zip.Stuff at the top of the source that is used to update:
Imports System.Net
Imports ICSharpCode.SharpZipLib
Public Class Form1
[code]......
View 4 Replies