Systems Integration
Project page for IT490 - Systems Integration
Summary
In this project, we designed and implemented a secure, multi-server distributed application that integrates a third-party data source into a custom web platform. The system simulates a cryptocurrency exchange using CoinCap’s API, enabling users to view market data, manage portfolios, and execute simulated trades. The project emphasized reliable inter-service communication, security controls, and operational monitoring in a Linux-based environment.
I was primarily responsible for backend development: I created the API calls (internal and external), set up the messaging system, and wrote the message queue listeners. I primarily used PHP, as the seed project was written in PHP.
Code Repository: https://github.com/Mike-ui-bot/rabbitmqphp_example/tree/final
Learning Outcomes
Applied skills across different domains of IT (full stack web development, systems administration, security)
Gained practical understanding of distributed systems and event-driven architecture
Learned how to design and operate a distributed system using message queues for asynchronous communication
Technology Stack
Operating Systems: Linux (Ubuntu 22.04 & 24.04 LTS)
Languages & Scripting: PHP, Python, JavaScript, HTML/CSS
Infrastructure & Services: RabbitMQ (AMQP), Apache2, MySQL, Node.js
Security & Monitoring: OpenSSL, Nagios, OpenVAS, Twilio (2FA)
Version Control: Git
Scripts
These might provide more context when you read through the deliverables.
“listener” and “handler” are often used interchangeably, but they work pretty much the same. These scripts listen on a queue, receive a message, and perform an action based on the message contents. The terminology at the time was confusing and we did not bother to rename the scripts.
Database/db_handler.php: Running on the backend VM, this script handles internal API calls such as user authentication, displaying coin data from the DB, and most of the functionality with the crypto site (buying and trading, viewing portfolio, etc.). This script listens for messages on db_exchange.
https://github.com/Mike-ui-bot/rabbitmqphp_example/blob/final/Database/db_handler.php
Database/crypto_handler.php: This is not really a listener, more of an API call that gets executed at intervals. This script sends a message to dmz_exchange with action getTop100Crypto. Once the DMZ handler receives this message, it will return the crypto data pulled from the API, back to the backend VM. The backend VM will then store or update the crypto data on the database.
The DB VM is not directly communicating with the DMZ. RabbitMQ acts as the middleman and forwards the messages to the appropriate client.
https://github.com/Mike-ui-bot/rabbitmqphp_example/blob/final/Database/crypto_handler.php
DMZ/dmz_handler.php: This script runs on the DMZ VM and listens for messages on dmz_exchange. It handles external API calls to the CoinCap API, such as retrieving data for the top 100 crypto, viewing historical price data, etc.
https://github.com/Mike-ui-bot/rabbitmqphp_example/blob/final/DMZ/dmz_handler.php
Logger/log_listener.php: This script would run on every system in the cluster. The script listens on a fanout exchange, log_exchange, receives log messages, and appends log messages to the appropriate file.
https://github.com/Mike-ui-bot/rabbitmqphp_example/blob/final/Logger/log_listener.php
Deployment/*: See the deployment demo in Final
https://github.com/Mike-ui-bot/rabbitmqphp_example/tree/final/Deployment
webserver/dbCryptoCall: This script sends a getTop100Crypto request to db_exchange from the frontend, and is executed whenever the dashboard page is refreshed. This retrieves crypto data from the local DB instead of the API so that we aren’t spamming API calls and using up our credits.
The webserver VM is not directly communicating with the DB. RabbitMQ acts as the middleman and forwards the messages to the appropriate clients.
https://github.com/Mike-ui-bot/rabbitmqphp_example/blob/final/webserver/dbCryptoCall.php
webserver/dmzCryptoCall: This script sends API requests to the DMZ from the frontend. These requests involve data that is not stored in the local DB, such as historical price data and additional coin information used to generate charts and graphs.
Once again, the webserver VM is not directly communicating with the DMZ. RabbitMQ acts as the middleman and forwards the messages to the appropriate clients.
https://github.com/Mike-ui-bot/rabbitmqphp_example/blob/final/webserver/dmzCryptoCall.php
webserver/crypto-side/notifications.php and check_price.php: These scripts work together and power our notification system. See Midterm
On the notifications tab of the website, the user enters the symbol of the coin they want price updates on, along with their email. The webserver passes these 2 values as arguments to check_price.php.
check_price.php is a background process running on the webserver VM that sends API requests (get_coin_price) every 30 seconds to the DB handler. Upon receiving the request, the DB checks to see whether the price of the coin changed. If it did, it sends the new coin price back to the webserver VM. check_price.php then sends the email with the new coin price to the user. If the price of a coin does not change however, then no email is sent.
https://github.com/Mike-ui-bot/rabbitmqphp_example/blob/final/webserver/crypto-side/check_price.php
Deliverables
A showcase of our deliverables in action with some context on how they work, if necessary. Our group missed one deliverable and got half credit for another deliverable, tallying the score to 25.5/27, or ~94/100.
Common deliverables - deliverables common to every group
Personal deliverables - unique deliverables assigned to a group based on the project proposal and data source used
Midterm
Up to the midterm, the server architecture consisted of one virtual machine hosting the webserver (Apache2), one virtual machine hosting a message queue server (RabbitMQ), one virtual machine hosting a database management system (MySQL), and another machine dedicated to pulling data from the API (DMZ). This DMZ was isolated from the webserver and DB VMs, however it can still communicate with the RabbitMQ VM.
At this phase of the project, I was administering the RabbitMQ and DMZ VMs.
Slide credit: DJ Kehoe
Common
- Functional website
- Secure database (no remote access)
- Inter-server communication through a message queue (RabbitMQ)
- Data collection through code (on demand or cron)
- Firewalls
- Authentication
Personal
- Ability to browse and see coin information
- Trading feature: ability to buy and sell coins (no real money is involved)
- Portfolio view with evaluation of portfolio, show gains/losses, etc.
- Push notifications (email)
- Recommendations based on position and volatility
- RSS feeds with crypto news
Final
For the final, we had to create clusters for different stages of a software development environment: Dev, QA, and Prod. Due to the amount of VMs needed to achieve this, we were allowed to combine the RabbitMQ and DB VM into one VM, now referred to as the Backend VM. Additionally, the DMZ is not really a DMZ anymore since it is placed on the same network as the webserver and backend VMs, so there really isn’t any isolation. The DMZ is pretty much just a data source processor.
A system on one environment should not be visible to another system on a different environment. For instance, the QA backend VM should not be able to communicate with the Prod backend VM. We achieved this isolation by placing the machines on different VPNs. So, Dev got its own VPN, QA got its own VPN, etc. However, the deployment VM must be connected to all 3 networks to support the deployment system.
At this phase of the project, I was administering the deployment system VM, the Nagios VM, and OpenVAS VM.
Slide credit: DJ Kehoe
Common
- Systemd - custom software running on systemd
- Deployment System - version control, deploy, rollback
- Production, QA, and Development cluster
- Host Stand By (HSB) System for Production (incomplete)
- Database Replication - ONLY between Primary & HSB on Prod (half complete)
- Hashed passwords for users in DB
- Decentralized Logging
- SSL - use of secure socket layer for web server (HTTPS)
- Responsive Website Design
Personal
- SMS/ text updates
- Discussion board/message board/forum feature
- Comparison feature - side-by-side comparison view of different coins
Extra Credit
These deliverables focused on enhancing the security of the application.
- 2FA
- OpenVAS - Security report for Prod cluster
- Nagios - Monitoring software for Prod