LATEST NEWS

Parsing Google search without api

img
Jul
23

hey guys, just wanted to share how you can parse google result without using google api…its a simple version.

You will be needing  HTMLAgilityPack to simplify parsing of HTML nodes. (https://htmlagilitypack.codeplex.com/)

    Function carigugel(ByVal query As String)
        Dim gs As New Net.WebClient
        gs.Headers.Add("User-Agent", ua)
        Dim res As String = gs.DownloadString("https://www.google.com.au/search?q=" & query)
        Dim doc As New HtmlDocument
        doc.LoadHtml(res)

        For Each link As HtmlNode In doc.DocumentNode.SelectNodes("//*[@id=""rso""]")
            Dim k As HtmlNodeCollection
            k = link.SelectNodes("li")
            Dim l As Integer = 1
            For Each item As HtmlNode In k
                item = item.SelectSingleNode("div/h3/a")
                Debug.Print(item.Attributes("href").Value)
                item = item.SelectSingleNode("//*[@id=""rso""]/li[" & l & "]/div/div/div/span")
                Debug.Print(item.InnerText)
                l = l + 1
            Next
        Next
    End Function

You can addtionally add other functions depending on the search page structure, eg. total result, next page,.. have fun!