# Experiment 7 – Log Analysis Using ELK or Grafana Loki

Objective: Analyze web application logs using ELK Stack or Grafana Loki to identify service failures, calculate HTTP error rates and visualize errors over time.

## 1. What is log analysis?

Log analysis is the process of examining system or application logs to identify errors, abnormal activities and performance issues.

It helps administrators troubleshoot problems, detect failures and understand application behaviour.

## 2. What is the ELK Stack?

ELK is a combination of three open-source tools used for centralized log collection, processing and analysis.

- Elasticsearch: Stores, indexes and searches log data.
- Logstash: Collects, processes and transforms log data.
- Kibana: Provides dashboards and visualizations for analyzing logs.

## 3. What is Grafana Loki?

Grafana Loki is a log aggregation system used to collect, store and query logs from different applications and systems.

It is designed to work with Grafana for searching logs and creating visualizations.

## 4. What is the difference between ELK and Loki?

- ELK: Uses Elasticsearch for indexing and searching log content.
- Loki: Primarily indexes log labels and stores log contents in chunks.

Both help centralize, search and analyze logs.

## 5. What are the main components of Grafana Loki?

- Loki: Stores and processes log data.
- Promtail: Collects logs and forwards them to Loki. (Other log collectors can also be used.)
- Grafana: Displays and visualizes log data.

## 6. What is log ingestion?

Log ingestion is the process of collecting logs from different sources and transferring them into a centralized logging system for storage and analysis.

For example, collecting web server logs from multiple application servers.

## 7. What is log filtering?

Log filtering means selecting specific log entries based on conditions such as timestamp, hostname, severity or HTTP status code.

It helps identify relevant errors without examining every log entry.

## 8. What are HTTP status codes?

HTTP status codes indicate the result of a client's request to a web server.

| Code range | Meaning       | Example |
| ---------- | ------------- | ------- |
| 1xx        | Informational | 100     |
| 2xx        | Success       | 200     |
| 3xx        | Redirection   | 301     |
| 4xx        | Client errors | 404     |
| 5xx        | Server errors | 500     |

Important: 4xx indicates client-side errors, while 5xx indicates server-side errors.

## 9. What is HTTP error rate? How is it calculated?

HTTP error rate represents the percentage of requests that result in errors.

\\[ \text{Error Rate} = \frac{\text{Number of Error Requests}} {\text{Total Requests}}\times100 \\]

Example:

- Total requests = 1000
- Failed requests = 100

Error rate = 10%.

## 10. What is the difference between 4xx and 5xx errors?

- 4xx errors: Client-side problems, such as invalid requests or missing resources.
- 5xx errors: Server-side problems, such as internal server errors or service unavailability.

Examples: 404 (Not Found), 500 (Internal Server Error), 503 (Service Unavailable).

## 11. What is a timestamp in log analysis?

A timestamp records the exact date and time when an event occurred.

It helps administrators arrange logs chronologically, identify when failures occurred and correlate events across systems.

## 12. What is log visualization?

Log visualization represents log data through graphs, charts and dashboards.

It helps identify patterns such as increasing errors, repeated failures and periods of maximum application downtime.

## 13. What is the difference between logs and metrics?

- Logs: Detailed records of individual events, such as failed login attempts or HTTP errors.
- Metrics: Numerical measurements collected over time, such as CPU utilization, memory usage and request rate.

Logs explain what happened, while metrics help measure system performance.

## 14. How would you identify the period of maximum failures?

I would filter application logs by HTTP error status codes and group them according to timestamps.

Then I would compare the number of errors across different time intervals to identify the period with the highest failures.

## 15. Why is centralized log analysis important?

- Helps troubleshoot application failures.
- Provides logs from multiple systems in one location.
- Identifies recurring errors.
- Helps detect suspicious activities.
- Supports performance analysis through dashboards.

## Last-minute revision

| Term          | Remember                               |
| ------------- | -------------------------------------- |
| ELK           | Elasticsearch, Logstash, Kibana        |
| Elasticsearch | Stores and searches logs               |
| Logstash      | Collects and processes logs            |
| Kibana        | Visualization and dashboards           |
| Loki          | Log aggregation system                 |
| Promtail      | Collects and forwards logs             |
| Grafana       | Visualization tool                     |
| Log ingestion | Collecting logs                        |
| 2xx           | Success                                |
| 3xx           | Redirection                            |
| 4xx           | Client errors                          |
| 5xx           | Server errors                          |
| Error rate    | Failed requests / Total requests × 100 |
| Timestamp     | Time of event                          |

Important for practical viva

Focus on:

1. ELK architecture and components.
2. Grafana Loki and its components.
3. ELK vs Loki.
4. Log ingestion and filtering.
5. HTTP status codes (especially 404, 500, 503).
6. HTTP error rate calculation.
7. Logs vs metrics.
8. Identifying maximum failures using timestamps.

Based on Experiment 7 of your CS414 lab question bank.&#x20;

CS414_ITIMM_Lab_ESE.pdf
