# Alarm Detection System

## Description
The Alarm Detection System is an advanced machine learning solution designed to distinguish between **Relevant** and **Irrelevant** alarms in network elements. This system provides real-time alarm classification, helping network operators focus on critical issues and reduce noise from non-essential alerts.

## Features
- **Intelligent Alarm Classification**: Utilizes a CatBoost Classifier to categorize alarms as Relevant or Irrelevant.
- **RESTful API**: Offers a robust API for seamless integration with existing network management systems.
- **Real-time Detection**: Provides instant alarm classification upon receipt.
- **User-friendly Interface**: Includes a Streamlit-based web application for easy interaction and data visualization.
- **Data Logging**: Automatically saves user inputs and model predictions in the `data/external` directory for future analysis and model improvement.

## Prerequisites
- Docker and Docker Compose
- Make (optional, for using Makefile commands)

## Installation and Usage

### Quick Start
To install the Docker Image:
    
```bash
make build
```

To run the application:

```bash
make dc-up
```

These commands build the Docker image, starts the containers, and runs both the FastAPI server and Streamlit application.

To stop the application:

```bash
make dc-down
```

### API Usage
The API endpoint for predictions is available at `http://127.0.0.1:8000/predict`. 

Example API request:
```bash
curl -X POST "http://127.0.0.1:8000/predict" -H "Content-Type: application/json" -d '[
  {
    "Severity": "Critical",
    "Alarm Name": "NodeB Unavailable",
    "Technical ID": "22214",
    "Site Name": "Dummar",
    "Site ID": "12345",
    "NE Name": "NodeB_123",
    "Source": "SourceA",
    "NE Type": "TypeA",
    "Technology": "M2000",
    "Alarm Category": "CategoryA",
    "Parent Node": "NodeParent",
    "First Occurence": "12/01/2015 02:41:22 PM",
    "Last Occurence": "12/01/2015 02:41:22 PM"
  }
]'
```
> Note: It can take more than one JSON object


### Web Application
Access the Streamlit web application at `http://localhost:8501`. This interface allows you to:
- Enter alarm data manually for prediction
- Upload a CSV file containing multiple alarms for batch prediction

<img src="streamlit_app/img/app.png">

## Model Training and Management

The Alarm Detection System uses a CatBoost Classifier, which can be retrained on new data as needed.

### Data Flow
1. Raw data is stored in `data/raw`
2. Processed data is saved in `data/processed`
3. The trained model is stored in the `models` directory

### Training Process
To retrain the model on new data:

1. Place new raw data in `data/raw` (maintain the same filename format)
2. Remove the existing CatBoost folder from the `models` directory
3. Execute the following commands:

```bash
make exec-in                # Enter the Docker container
make process-raw-data       # Process the new raw data
make process-training-data  # Prepare data for training
make train-model            # Train and fine-tune the model
```

The `train-model` command performs the following actions:
- Trains the model using the processed data
- Fine-tunes the model to find optimal hyperparameters
- Saves the best hyperparameters in `configs/model_training/<MODEL_NAME>.yaml` under `best_params`
- Saves the new model in the `models` directory (if it doesn't already exist)


## License
This project is licensed under the GNU GENERAL PUBLIC LICENSE. See the [LICENSE](LICENSE) file for more details.