Counting the IP addresses hits to your website or application with Go Lang.

Hello Folks, Recently I was working to find out the count of IP addresses which hits to website. The logs files generated by application.

The result can be achieved with help of various ways but I thought How it can be achieved in the Go Lang?

So the entire code with sample application logs pattern can be found here.

I have considered the logs pattern would be fixed, or the in one log line the IP addresses would be occurred once. I had considered the custom logs and apache access logs and for both of the logs pattern program works fine.

I had considered two scenario, in first scenario program would except to enter the IP address and it will display how many times the IP address occurred in the log file and in other scenario one will run the program and it will display all the IP addresses and the number of times the IP addresses occurred found in log files.

We will discuss here the first approach. Refer the “main.go” file. You need to enter the IP address that you want to search and it will display result as below. Obviously the logs file you need to keep in the same directory and need to refer the same in “main.go” file.

Input provided by the User

We had used the “os” “bufio” “strings” and “net” package.

First it will validate the IP addresses with the help of the “net” packages then it will open the logs file. With help of “ bufio” it will scan word by word and append it in our large slice of string.

Finally with help of “Count” function it gives result for how many times the particular IP address is occurred in log file.

           lgf, _ := os.Open(filename)
           scanner := bufio.NewScanner(lgf)
           scanner.Split(bufio.ScanLines)
           var text []string    //slice of strings
           for scanner.Scan() {
             text = append(text, scanner.Text()) //appending word by word in text slice
          }
          lgf.Close()
            for _, each_line := range text {
                  cnt = cnt + strings.Count(each_line, args[0])
            }
      fmt.Printf("The IP hit to our website %d\n" , cnt)

Another approach is to run the program and it will scan the entire log files and will find out distinct IP addresses. Display the result as IP addresses and the number of times it hits to the websites as below.

Showing the IP addresses with number of times hit to website

First defined the regex pattern for the validation of IP addresses then open the file, scan the file with help of “bufio” package. After that split into lines so that each line of log file is treated as single string. Defined variable and get “submatchshell” with help of “findAllString” function which belongs to strings package.

Loop the same logic to extracts all the IP addresses. Append the all the IPs in one slice of string and passed the slice of type string to function. The function count the occurrences of IP addresses as below.

func countOccurrences( mapstr []string) {
       dict := make(map[string]int)  // defined one map for counting
      for _, v := range mapstr {
             dict[v]++   //If the same IP addresses occurred then its value increased by 1.
     }
     fmt.Println(dict)
     keys := make([]string, 0, len(dict))  //defined slice to store keys of maps
    
     for k := range dict {
        keys = append(keys, k)
     }
    sort.Strings(keys)  //sorting based up on keys
    for _, k := range keys {
         fmt.Println(k, dict[k] )  //finally printing key(IP) and its value(The number of times it occur in the log file
     }
}

Hope it will help to someone to count the IP address occurrences in logs file.

Thank you.