I'm just starting working on a program and the amount of pages I'm trying to screen scrape take over 20 minutes, so I was hoping I could run like 4 or 5 threads to cut that down??? I'm pretty much still a novice, so be easy on me. I understand good, though.
View 1 RepliesI am developing a web program using asp.net(vb) that scrapes data of a certain website. I am using System.Net.HttpWebRequest and System.Net.HttpWebResponse.My problem is I can not retrieve the codes of certain frame/container where the data that I needed is located. I mean, when I view the source code of the website, I can not find the data but I can see it on the web page. When I view source it, it is under the
[Code]...
I'm trying to develop a windows form app which will take a screen shot every 30 seconds for about 45 minutes. After 45 minutes, the user will click Finish, and these screenshots will be uploaded to a database. I don't want anything that simulates a PrintScreen key press, because I don't want to alter the users clipboard. The second issue is, for users with multiple screens, I'd like to capture the entire screen image, not just from the primary screen. I'm using Visual Basic .NET 2005.
View 2 RepliesI am looking for formula that is accurate in converting Minutes to Decimal Minutes i.e 30 minutes is 50 Minutes in Decimals so 1.3 hours is 1.50 and not 1.5 i have used Minutes/60 but it is not accurate as some times i have to Multiply the result by 100 and some time 1000 to get correct results based on single digit Try 1/60, 10/60 99/60 or 100/60.What would be the most efficient and correct way to Convert the time into Minutes ?
View 10 RepliesI'm trying to make a small scraper can't figure out how what i want to do is scrape the <a href over the webpage I just navigated with webbrowser1.navigate now there are many <a href over the page i need to scrape all the <a href only this ones:
"<a href="/page/page/218/445/"><img src="/images/***.gif" width="44" height="16" alt="Download ***" title="Download ***" border="0"></a></td>"
i need the code between "<a href=" and "><img is there a command to find a string in html after <a href=" and before "><img ? scrape all of them there are many and save it over txt file how can i do that?
I'm trying to make an app that will scrape numbers off of a webpage. What I want to do is have it read the Game Name and then Views (for statistics keeping). The WebPage is set up like
<tr class="odd">
here are 7 <td> tags that display different things
</tr>
[Code]....
I'd like the app to check the second TD tag to see if it's innertext says, lets say, 'GAME', and then if it does, it adds the innertext of the 7th TD tag (which is a number), to the total sum, and it scrapes all of that info off the page.
I can understand the logic of how to process the info, but I have no clue as to reading the correct tags.
Im trying to scrape some text on a webpage, I asked in the regex section and they recommended to use HtmlAgilityPack with Xpath to scrape the info I want.
[code]...
I am using a for next loop to scrape through some html code. I am testing elements for a certain string, and when it hits that, I need to get the string that resides 2 elements earlier.When going through a for...next loop (I know you can loop completely backwards with step -1), is there a way to 'go back' 2 loops?
Ex)for each'lets say we are 5 loops in and our if returns true'can i go back to loop 3, perform an action, then return to loop 5 and continue the real loops?
I'm trying to make a small scraper can't figure out how what i want to do is scrape the <a href over the webpage I just navigated with webbrowser1.navigate now there are many <a href over the page i need to scrape all the <a href only this ones:
"<a href="/page/page/218/445/"><img src="/images/***.gif" width="44" height="16" alt="Download ***" title="Download ***" border="0"></a></td>"
i need the code between "<a href=" and "><img is there a command to find a string in html after <a href=" and before "><img ? scrape all of them there are many and save it over txt file how can i do that?
I'm trying to scrape the right url from html file using webbrowser I want to scrape this Href and navigate to it. But the problem is every other comment with reply is almost the same. So if I use to scrape hrefs and check the name it will give me the reply buttons of all the comments + the new comment button. Is there a way to grab this link only this one by it's Class name or something?
<a href="forums.php?op=post&p=1409951"><img src="/images/icons/comment_add.png" class="inline_icon" align="top"> New Comment</a> The ones I don't need:
<a href="forums.php?op=post&p=1409971">Reply To This</a> I'm trying to create my own browser and this should be a button short cut If I want to comment.
I have an application that provides a countdown for 60 minutes and I'm currently using the win form timer and it updates some labels and a progressbar. Now I have to change this to be a countdown for 360 minutes. Will the win form timer be sufficient or should I implement a Timers.Timer instead?
View 6 RepliesI just got VB and I am having a hard time learning this stuff. but I am not giving up.I am looking to make a web text scraper, so I can scrape words off of webpages and put them into a text file.I couldnt find a whole lot of help in the search function. bare with me, I am new here and new to programing also.
View 5 Repliessee this codes scrapes all href links and check if it contains "/file/" to save it but I get duplicate links saved so If i can change this code to work some how with Innertext("More") I will have no duplicatestried to configure it to work with innertext it just doesn't fit the way I think it should ;/and if anyone can add how can I remove duplicated urls on my txt file that would be really nice I might need it
Dim links As System.Windows.Forms.HtmlElementCollection
Dim b As String
links = WebBrowser1.Document.Links
[code]....
I have used Web Browser in VB to get the HTML source code of a web page and put it in a richtextbox. I need to take that HTML and extract the data needed from it. I have searched and cant find an example that I can understand being new to VB.Net I am trying eventually import the data into excel.
[Code]...
I am *VERY* new to web-scraping and am trying to scrape some information off of a webpage that is heavily javascript enabled. An example of the page I am trying to scrape from is: [URL] I am trying to scrape the property links such as "322 E 98th St" The text appears on the webpage and I can find the link myself, but it doesn't appear in the page source code.
I am trying to scrape it using the webbrowser control using the WebBrowser1.DocumentText property, but it doesn't even show the links simply when I view the source in ie. I am sure this has something to do with the javascript it uses to load up the page or maybe iFrames,
BTW this issue occurs in any MS office program when the VB.Net (or even VBA) is processing information.Example: In Excel, a worksheet is displayed on the screen. I start either, a VB.Net or VBA procedure and within 30 seconds the Excel worksheet (previously displayed) blanks out. In both VB.Net and VBA,ScreenUpdating = False. My expectation is that the previously displayed screen would stay static as if I left my desk to get a cup of coffee; came back and the same ole Excel worksheet was still there?Of course, setting VBA Screen Updating = False accomplishes two goals: 1) speeds up processing and 2) saves the user from seeing unnecessary "garbage-processing" steps.Why does VBA or VB.Net ScreenUpdating = False not freeze the screen at the time of its invoking?
View 11 RepliesOk so basically heres what i need to do: Extract text from the webpage that meets a certain criteria. There will be a ton of these on 1 page and i would like to add them to a rich textbox on sperate lines.
I know that it needs to be in a loop and its needs to Parse the wepage(Dim web1 As String = Me.WebBrowser1.Document.Body.InnerText)
The criteria is: Starts with 1 to 4(random) integers, Followed by "my" then 13(random) numbers and letters. Or if it starts with "167my" + 6(random) number and letters.
Edit: Also im going to try to make it loop through a list of webpages to do this.
imdb scraper for my movie organiser. Ok so how do you scrape movie ratings from imdb from a movie title name.
View 22 RepliesCurrently I have the following VB.NET code to make a screenshot of my desktop, but it only takes a picture of the active screen:
Public Function SaveScreen(ByVal theFile As String) As Boolean
Try
SendKeys.Send("%{PRTSC}")[code].....
The following code is how I execute the above function, if it makes any difference, which I don't think it does:
SaveScreen("C:Lexer_tracescreen.png")
Now, I need to be able to take a picture of the entire screen, not just the focused window.
Now I'm creating at app in VB (Microsoft's, Visual Basic 2010) which will be in full screen but I want to know if I can put all my content in the centre of the screen. At the moment it's at the far top, left of the screen. When the screen size varies I want it will stay in the middle for all shapes and sizes. Like :
<div style="margin: 0 auto; width: 500px;"></div>
But this is for web pages using HTML and CSS.
I looked around the forum can't find something really simple.I want to scrape everything between the
<!-- google_ad_section_start -->
to
<div class="ad_editorial-sponsorship"></div>
copy it and use it right after it's website and I navigate using webbrowser1.
Is there a way to have VB to use all available RAM in less than 3 minutes? say on a system with 4gb
View 4 RepliesI'm trying to get coordinates on my screen by holding down my mouse and placing it over different parts of the screen. I'm using the mouse_up and down events in a button to kinda simulate the mouse being held down. The problem I'm running into is that anytime I go past the button's bounds, the coordinates I want stop. This is what I have so far:
[Code]...
Sometimes when updating the entire screen or moving around controls etc, there is some fairly bad screen flickering.
View 2 RepliesI have created a GUI and have set my 'Form1' window to match that of my PC monitor ( 1280x1024 ). However when i move the project to work on my netbook, the screen on my netbook is only 1024x600 and half my GUI is missing. Is there any way i can adjust my settings so that my 'Form1' GUI will auto adjust and display in full on both screen when i move the project back and forth?
View 2 RepliesI'm coding a splash screen in VB.Net that displays for 3 seconds then shows a login screen. But the splash shows up even when login shows and I have told the splash to hide. Here is my code:
[code]...
I send a message a while ago and no one answerd, how can I draw directly on the screen (not on a specific window) just on the screen, neither if it's on the desktop or anything else.Is there is any option to bring up the switch between applications window (Alt + Tab), I don't want to use sendkeys because the user need to keep the key down and I just want the user to select the application with his mouse.
View 3 Repliesim about to develop a small appliacation for a bet system, which needs 6 screens, each screen will be a touch screen for each client... the application will be running in one computer only just with 6 screens. my question is does anyone knows how to control each screen separately?. somethign that would allow me to output each forms in different screens would work fine.
View 2 RepliesI am making a program that will put a black mask over an application (like a full screened game). This black mask will fade in and out, depending on certain criteria. Is there a way I can do that with VB? It's to help me dim my monitor without having to push anything.
View 3 RepliesI wanted to make a screen recorder (not screen capture) program, but I am not entirely sure how to get it started. I have seen many videos and things on how to make one, but all of these just take multiple pictures and then don't compile them into a movie file, and I don't want to take many pictures (unless that is the only way possible.) I am using Visual Basic 2010 Express and I have looked at the Windows Media Encoder, but I can't seem to figure out how to use/implement it (yes, I have downloaded and installed it.) Maybe I can use some kind of ActiveX control?
View 3 Replies