I was amazed with how little documentation or examples are out there for checking the pagerank for a website using vb.net . I was hoping that someone might have some example code or other resources regarding how to do this (using toolbarqueries.google.com - not by scraping data from a 3rd party website).
View 3 Replies
I have a windows forms application that scrapes a website to retrieve some data. I would like to implement the same functionality as a windows service. The reason for this is to allow the program to run 24/7 without having a user signed in.
To that end, my current version of the program uses a web browser control (system.windows.forms.webbrowser) to navigate the pages, click the buttons, allow scripts to do their thing, etc. I cannot figure out a way to do the same without the web browser control, but the web browser control cannot be instantiated in a windows service (because there is no user interface in a web service).
I know how to browse page witht he browser control etc, but i need to know how to grab data from within the html.
[Code]....
I want to read the data from the following website:[URL]..I have attached an image the details, from which I wish to read the data: getting the details as mentioned above?
And also let me know how the data is supposed to be extracted every time when required.I will be using this XML data in VB10. This will simply allow me to create a widget that will show me the cricket score update that frequently.
I have a website where people do the data entry. I have data stored in SQL server.Now I need to push the data to 3rd party to do some analysis work.What are my options to do this kind of job?
View 1 RepliesI have browsed through many posts on this and have tried some of the suggestions but still not understanding it fully. I would like to scrape html pages that have some script running that usually executes the script to display a link after clicking. Some mentioned firebug and others talked about reverse engineering the code I need. But after trying reverse engineering I still dont see how to get the data after tracing the script function.[code]I am using vb.net and some sites were easy using firebug where looking at the script I was able to pull the data that I needed. What woudl I do in this scenario? the link is url... and the categories are what I am trying to access. Notice the url does not change.
View 1 Repliesi am trying to parse an excel file. its structure is very complex. The possible way i know are.
Use Office introp libraries
Use OLEDB provider and read excel file in a dataset.
But the issue is of its complexity like some columns,cells or rows blank etc. What are the best possible ways to do this?
i need to scrape a p tag which has h3 tag after it but does not have a closing p tag. It looks like this :
<script ad>asdasdasd</script>
<p>Translation companies are
-----------------------
-----------------------
<h3 class="this_class">mind blown site</h3>
There is no </p> tag so i cannot parse it completely. Now i have two questions :
1) can this be parsed using httpagility xpath ?
2) i have a function to find text between two strings (getbetween). But i have a doubt - If i use "asdasdasd" and " is it always 100% that vb.net will use the script tag which is just above h3 because there are 2-3 same lines - "asdasdasd"
working on some code that will screen scrape a site then fine some code in there and pull it out for me...but for some reason i cant find the right regular expression. Anyone can with creating a regular expression that will match
/clipmark/C51BC80B-173F-4BC7-B630-CCDB8B56C9DF/
Have tried multiple attempts of writing it and each time it doesnt bring back all of the content or the wrong content.
Im trying to get school information of a site and want to save it as an excel tables sheet with details in each column, inital start was the below code help me in going further.Column headers: school name, mascot, address, type, phone, fax etc. for a list of schools I have. For example I used one link.
[Code]...
Is there anyway to access internet sites if i code in VB6. i have gone thru a few basic tutorials and didn't found any command/method to develop or code a program which can access internet and do my work.
View 6 RepliesIm making a vb.net app and im using htmlagilitypack. I need hap to get the profile links from [URL]
Here is an example of the html:
<a href="/bus/Ontario/Brampton/A-Safe-Self-Storage/17142.html?what=af&where=Ontario&le=1238793c7aa%7Ccf8042ceaa%7C2ae32e5a2a" onmousedown="utag.link({link_name:'busname', link_attr1:'in_listing_left',
[Code].....
I am making scraper that uses url list. But I have no idea how to pass that wanted url from that list to thread that does scraping.
Public Sub init2_check()
Dim t(source1.Items.Count()) As Thread
For x = 0 To UBound(t) - 1
t(x) = New Thread(AddressOf ScraperDoWork)
t(x).IsBackground = True
t(x).Start(x)
thread.Sleep(100)
Next
Me.Invoke(New button_Delegate(AddressOf enable2_button), btn_StartScrape)
End Sub
I want to download a webpage by httpWebRequest.
Web page Address >> http://www.stockbangladesh.com/resou...l=18455&inv=60
from a free website
[URL]
But that page requires login first. my user name : babul37 and pass : mhbb7337 How can I download that page by httpWebRequest?
I have just made a program were i enter a proxy list then my software views a url over and over using the proxys . I was using a web browser in side the program but worked out slow.So i now have changed the code to htpp request. But now when i run the program it goes up by 2 the views then frezzes .Here is my code
rivate Sub Timer1_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer1.Tick
If ListBox1.Items.Count = 1 Then
[code].....
create website involving payments plus some data entry eg booking of some facilities with the main data kept at a LAN window system.
View 4 RepliesI need to scrap data from a website where data is Inside table.
View 1 RepliesDaily I go to a site input my username etc and obtain a canned report. Is it possible to have code access the site get the report?I have code that manipulates the file however I was wondering if there was a way to do this first step in lieu of the manual process I am currently doing.
View 2 RepliesI have an XML with following structure[code]...
What will be easiest way using vb.net to get description for each SIC from the website? I can not modify this XML file. Do I need to create a collection with SIC code and description manually copied to it and then get description of SIC code in XMl by matching SIC code with collection ? Final collection will be used to populate a repeater with each item/row having SIC code and Industry Name.
how to make a VB program where, it'll have 2 textboxes, one username and one for password, and when the user enters in the information, it places it in the corresponding box on the website.
I'm making a program for a friend and his website, and this program is designed to allow the users of the site to download it, and use it to check if there are any updates (Friends, Mail, Forums, etc).
The code I have right now (which I tested to checking my IP) is:
Dim req As HttpWebRequest = WebRequest.Create("http:whatismyip.com/automation/n09230945.asp")
Dim res As HttpWebResponse = req.GetResponse()
[Code].....
I have a webbrowser that loads a long string of text data from our sebsite. I want to extract only a certain data from the webpage. The only thing it sometimes is not in the same place in the code. The only thing standard is the id= before the data i want. sample of how the data listed: Id=00234.08.
View 1 Replieshow to find something on a webpage and then put it into my vb form. Like pulling the name of someone from [URL] and posting it into a textbox.
View 2 Repliesi wanting to display the names in textbox of the site the code is <a class="big" href="KeepTheFaith">KeepTheFaith</a>there wil be diffrent usernames im wanting to extract into listbox can any one help me out,as iv only done form filling before not extractin data..im using webbrowser control i just want to grab the username then display in listbox any idea how to grab the hred="username"
View 6 RepliesI need to display the MinAH price and the name of the item listed in the table. If possible, I would also like to make a listbox of all the servers/realms in World Of Warcraft, giving the user the ability to change the realm that they want from the list because the prices will be different. Another thing would be a specific item instead of the ones already listed in the initial table (see link if you don't understand). You can search items on the website, I'm not sure how I could incorporate this so the user can select a specific item.
View 1 RepliesI want to retrieve data from a this link:
[URL]
I have used this code:
Private Sub WebBrowser1_DocumentCompleted(ByVal sender As System.Object, ByVal e As System.Windows.Forms.WebBrowserDocumentCompletedEventArgs) Handles WebBrowser1.DocumentCompleted
[Code]....
i want to search some data from some sites . if the dta is found it should be shown in a gridview with the urlfrom which it is found.
View 1 RepliesI am wanting to get data from a website to use in my calculations, the website is:
[URL]
I then want to (Imput, the station ID) for example "OMDB" then automatically click the button to get the required data. once I have this I need to get parts of this data into a textbox to complete my calculations.
I have little knowledge of this process but understand I need to use the Ellements of the page to get the data I want.
So this is what I have at the moment, but it is no realy getting what I what:
Public Class Form1
Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
On Error Resume Next
[Code].....
All I've been able to do is create the form. We have to get information off of a website and load certain information back into the textboxes on our form.
Assignment 6 - Text Parsing Many applications require you to extract information from a block of text. We will be using this heavily in the project. This project is designed to give you some practice extracting information out of text files. You will need to open up a file, search the file for some specific content, and then copy that content and place it on screen.
Go to Jobs.com and select a state that starts with the same letter as your last name (if there is no state that starts with the same letter, use the second letter, then the third letter, etc.). For example if my name was 'John Byway', since there was no B or Y, I would pick a state starting with W - either Washington, West Virginia, Wisconsin, or Wyoming.
The idea is you want to extract all the jobs information out of the page. We want to put the jobs names in a combo box, so the use can pick any of the jobs. Ideally, this would also show the information about each job. You don't need to do that. However, you do need to extract the information and put it on the screen (When you load the next job, the old job information will be lost, and you will be left with the information about the last job loaded)
Tasks
1. Go to the above address, view the source in the browser, copy and paste it into a text file. You do not need to access the Source of the web page within your application.(in IE go to View / Source; In FireFox go to View / Page Source)
3. Extract each job title and place the name in the Combo box.
4. Find and extract the following fields. Note, some jobs may not list all of these. In that case, get as many as are presented.
a. Date
b. Title of job
c. Company
d. Location
[code].....
Indicate on screen how many jobs you found. Note, you only need to look on the first page of jobs?
This website "[url]..." contains currency exchange rate, and it's useful to my desktop application " bill system " .the question is: How to get data from this website to my desktop application textboxes ?
View 4 RepliesI am using vb.net and I want to save data of shopping cart after i close the website on when I go and re-run of shopping cart is still there. Edit Edited the answer in here. This is for anyone who have queries on where to place the codePlace this code in web.config
<profile>
<properties>
<add name="PostalCode" />
[code].....
