dark-web-monitoring-using-nlp

Cyber Threat Intelligence System is an interactive Streamlit application for analyzing suspicious text and URLs. It combines NLP preprocessing, TF-IDF feature extraction, and an XGBoost classifier to estimate safe and threat probabilities, confidence scores, threat categories, and severity levels. The dashboard also calculates a threat intelligence risk score using factors such as VirusTotal malicious detections, machine learning confidence, and trusted-domain verification. Key features include a threat analysis interface, session-based threat logs with CSV export, visual analytics for threat distribution and probability trends, and a cybersecurity intelligence feed from The Hacker News RSS that can be analyzed through the same ML pipeline. URL inputs can optionally query VirusTotal for additional intelligence when a valid API key is configured. The project is intended for educational and cybersecurity analysis purposes. It uses Python, Streamlit, Pandas, NumPy, NLTK, Scikit-learn, XGBoost, Matplotlib, Feedparser, and Pickle. Installation follows a standard Python workflow: clone the repository, create and activate a virtual environment, install dependencies from requirements.txt, configure the VirusTotal API key securely, and run the app with Streamlit.