Parsing Google search without api

hey guys, just wanted to share how you can parse google result without using google api…its a simple version.

You will be needing  HTMLAgilityPack to simplify parsing of HTML nodes. (https://htmlagilitypack.codeplex.com/)

    Function carigugel(ByVal query As String)
        Dim gs As New Net.WebClient
        gs.Headers.Add("User-Agent", ua)
        Dim res As String = gs.DownloadString("https://www.google.com.au/search?q=" & query)
        Dim doc As New HtmlDocument
        doc.LoadHtml(res)

        For Each link As HtmlNode In doc.DocumentNode.SelectNodes("//*[@id=""rso""]")
            Dim k As HtmlNodeCollection
            k = link.SelectNodes("li")
            Dim l As Integer = 1
            For Each item As HtmlNode In k
                item = item.SelectSingleNode("div/h3/a")
                Debug.Print(item.Attributes("href").Value)
                item = item.SelectSingleNode("//*[@id=""rso""]/li[" & l & "]/div/div/div/span")
                Debug.Print(item.InnerText)
                l = l + 1
            Next
        Next
    End Function

You can addtionally add other functions depending on the search page structure, eg. total result, next page,.. have fun!

yusri

He has always been interested in computing. He is very passionate when it comes to technology oriented knowledge. He knows that the more that one knows, the more that one needs to learn. Experienced in content management and forms processing, he was a FileNet and Datacap Specialist. He is also a part time gamer and streams sometime.