README.md

@@ -1,84 +1,3 @@
-# Imperial College London
-# Sensing and IoT Coursework Project
-#### Reducing water waste by automatically dispensing lower bound water amounts, calculated using a ML Model using present weather data
+# ICL-IoT-Weather
 
-## About
-This project aims to integrate hardware IoT sensors (some RPi Zero's with I2C probes) and Machine Learning (Deep-NN) to predict how much water a outdoor garden will need every day.
-
-The ML Model uses the last 24hrs of collected weather data to make a prediction around mow much water needs to be dispensed to keep the plants alive.
-
-This approach hopes to reduce the overall water use by not watering plants in excess.
-
-## Project diagram
-```TODO```
-
-## File Structure
-
-<details>
-<summary>This part is long, click to expand</summary>
-
-- `data_collector/` Main data collector and uploader module
-  - `secrets/` Private API keys
-    - `icl-iot-weather-firebase-adminsdk.json` Firebase key for database access
-    - `weather_api_key.txt` OpenWeather API key
-  - `data_colelctor.py` Python script for data collection
-  - `Dockerfile` Containerizing the application
-  - `requirements.txt` Python requirements for running the script
-- `data_processor/` ML on demand data processing module
-  - `firebase_python_image/` Since building a tf image takes ages, we build it once and use it for testing the ml script later
-    - `Dockerfile` Building a tensorflow image for armv7l
-  - `secrets/` Private API keys
-    - `icl-iot-weather-firebase-adminsdk.json` Firebase key for database access
-  - `watering_model.model/` Saved ML modelfor water predicitons
-  - `Dockerfile` Containerizing the application
-  - `water_predictor.py` On demand, real time watering predictor script
-- `Diagrams/` Process and block diagrams
-  - `block_and_process_diagrams.drawio` TODO
-- `lora_nodes/` Scripts running on the Pi Zero nodes
-  - `master/` Scripts running on the primary, internet connected Pi
-    - `irrigator/` Daily watering module
-      - `Dockerfile` Containerizing the application
-      - `irrigator.py` Daily watering script
-      - `requirements.txt` Python requirements for running the script
-    - `Dockerfile` Containerizing the application
-    - `main.py` Main data server, collects data from satellite over LoRa and returns over http
-    - `requirements.txt` Python requirements for running the script
-  - `satellite/` Data reading and sending module for the sensor Pi
-    - `Dockerfile` Containerizing the application
-    - `main.py` LoRa commend listener, temperature reader and pump controller
-    - `requirements.txt` Python requirements for running the script
-- `manual_data_processing/` iPython notebooks used for data processing and model training
-  - `datasets/*` Various datasets used for processing and training
-  - TODO
-- `site/*` Monitoring website, HTML+CSS+JS, hosted on Firebase
-- `.gitignore` Gitignore file preventing all my API keys from showing up online...
-- `README.md` See `README.md`
-
-</details>
-
-## Installation and running
-1. Clone this repo
-2. Create the secrets folders
-3. Generate a set of firebase and OpenWeather API credentials and place them in the secrets folders, rename accordingly
-4. Move the `master` folder to the main Pi
-5. Move the `satellite` folder to the sensor Pi
-6. Install Docker on all Devices
-7. Build the docker images on the relevant architectures (for me everythong was build on armv7)
-   - `data_colelctor`: `docker build -t siot-weather-collector .`
-   - `data_processor`: `docker build -t siot-data-processor .`
-   - `master/irrigator`: `docker build -t master-node .`
-   - `master`: `docker build -t irrigator .`
-   - `satellite`: `docker build -t satellite .`
-8. *NOTE: The ML Dockerfile is very long and complicated because tensorflow does not play well with a 32 bit arm architeture, if you are building for x86 or arm64, you may need to change the file*
-9.  Run the docker containers:
-   - The `data_collector` on the cloud device: `docker run -d --restart always --name siot_weather_collector siot-weather-collector`
-   - The `data_processor` on the cloud device: `docker run -dp 3535:3535 --restart always --name siot_watering_predictor  siot-data-processor`
-   - The `master` on the master Pi: `docker run -dp 3333:3333 --privileged --restart always master-node`
-   - The `master/irrigator` on the master Pi: `docker run -d --privileged --restart always irrigator`
-   - The `satellite` on the sensor Pi: `docker run -d --privileged --restart always satellite`
-10. While you are free to use my ML model included in this repo, I suggest you explore the `manual_data_processing` folder and create your own.
-11. You will also need at least 24hrs of data before the model can make predictions, so the `siot-weather-collector` image must be started at least 24hrs before the others
-
-## Maintainers and Contributors
-
-   Max Hunt: [me@maxhunt.design](mailto:me@maxhunt.design)
+# TODO

data_collector/Dockerfile

@@ -4,4 +4,4 @@ COPY . .
 RUN pip install -r requirements.txt
 EXPOSE 80
 EXPOSE 443
-CMD "/code/data_collector.py"
+CMD "/code/Data_collector.py"

diagrams/e2e diagram.drawio

@@ -1 +0,0 @@
-<mxfile host="Electron" modified="2021-01-09T01:51:41.772Z" agent="5.0 (Macintosh; Intel Mac OS X 11_1_0) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/14.1.8 Chrome/87.0.4280.88 Electron/11.1.1 Safari/537.36" etag="gbp0hDtj5hE5ZAe5tMTm" version="14.1.8" type="device"><diagram id="QUbS_Z7dZOKaG96ER5AO" name="Page-1">7ZjbctowEIafhstkfMIhl4GQ0E4zTUtnuBZoa6uxJY+05tCnr4RlDJiC22nBdHqF/K800n76tdjq+IN0+SxJFr8ICknHc+iy4z92PM91nVD/GGVVKL3grhAiyWghOZUwZt/BjizVnFFQViskFCJBlu2KM8E5zHBHI1KKxW63ryKhO0JGIqgJ4xlJ6uqEUYxtFt5dpY+ARbGdudvzi0BKyr42ERUTKhZbkj/s+AMpBBatdDmAxLDbxfL0k+hmXRI4Nhmw4CIMsuebKet+ehu+fV6+5083drFzkuQ2X7tYXJUAgGoe9lFIjEUkOEmGldqXIucUzDSOfqr6fBAi06KrxW+AuLKbS3IUWooxTWy0norNTolczuDI+ktHEBkBHuln0zK5bE1gQT2DSAHlSneQkBBk8929J9ZC0aZfhVk3LOlfoO7WqH+BNOt4/VGeMsrQrEQBV0LWdqNibcAtYoYwzsia0UIfwIZc5yARlkdJlNHSvvb4uqF9XlSHwQ2sFm8dhFL74/C6V25Zr6Flg1ZZ1jtAPUz0+vtT3YhMY2wd67yyTUyWwVLRk28GPBIk6wQIhcs73euddnrvnEa/u3KjBw2NHrbK6MFpo78QhfBbRk/I6uI+9xv4/P6cPr+/cp+HDX3ea5XPy/fr/9jPi/3QH+m/iN11W8U9bFLWGTeTeI6p1wNNZq/Ib5f0euEfCYWMR2bxnJoP3qLoZ1LMQKl15MKVP7hv8C4fnrP0907vitmLKVHQbBOemISit2OaCoWEi3MP97n73Tp375zc3foXaA38BKYPWdYMu2ICb1OyjHOOtxQUi3j7qJc3M5ejHlxXqXf9hrW+XW/w7qFLrT1zf8yAT4BgvC7xD6/vmvm8GlJV9zlrw9er7++53fl79zT6sbq2XMe27n794Q8=</diagram></mxfile>

dnup_virtual_sensor/virtual_soil_sensor.py

@@ -1,4 +1,3 @@
-#!/usr/local/bin/python
 import logging
 
 import firebase_admin
@@ -10,17 +9,15 @@ from tensorflow import keras
 class Firebase:
     def __init__(self):
         self.creds = credentials.Certificate(
-            'secrets/icl-iot-weather-firebase-adminsdk.json')
+            'icl-iot-weather-firebase-adminsdk.json')
         firebase_admin.initialize_app(self.creds)
         self.db = firestore.client()
-        logging.debug('Initialized firebase instance')
 
     def pull_from_db(self, orderby=u'timestamp'):
         doc_ref = self.db.collection('weather_data')
         query = doc_ref.order_by(orderby,
                                  direction=firestore.Query.DESCENDING).limit(1)
         doc = query.stream()
-        logging.debug('Got doc file from firestore')
         return doc
 
     def convert_to_float(self, data):
@@ -40,14 +37,11 @@ class VirtualProbe:
     def __init__(self):
         self.firebase = Firebase()
         self.model = keras.models.load_model('virtual_probe.model')
-        logging.debug('Loaded ML model')
 
     def predict_soil_temp(self):
         feature = self.firebase.get_feature()
-        predicted_temp = float(self.model.predict([feature])[0][0])
-        predicted_temp_2dp = float("{:.1f}".format(float(predicted_temp)))
-        logging.debug(f'Predicted temp: {predicted_temp_2dp}')
-        return predicted_temp_2dp
+        predicted_temp = self.model.predict([feature])
+        return predicted_temp
 
 
 if __name__ == "__main__":

lora_nodes/master/irrigator/Dockerfile

@@ -3,4 +3,5 @@ WORKDIR /code
 COPY . .
 RUN pip install requests
 RUN pip install -r requirements.txt
-CMD "/code/irrigator.py"
+EXPOSE 3333
+CMD "/code/main.py"