No public description

PiperOrigin-RevId: 657786644

official/projects/waste_identification_ml/circularnet-docs/archetypes/default.md

@@ -0,0 +1,5 @@
++++
+title = '{{ replace .File.ContentBaseName "-" " " | title }}'
+date = {{ .Date }}
+draft = true
++++

official/projects/waste_identification_ml/circularnet-docs/content/_index.md

@@ -0,0 +1,21 @@
++++
+title = ''
+date = 2024-07-30T13:46:54Z
+draft = true
++++
+## CircularNet overview
+
+CircularNet is a free computer vision model developed by Google that utilizes artificial intelligence (AI) and machine learning (ML) to provide detailed and accurate identification of waste streams and recyclables. Trained on a diverse global dataset, CircularNet aims to make waste management analytics accessible and promote data-driven decision-making. It supports efforts to keep valuable resources out of landfills and in circulation.  
+Open access and collaboration are fundamental to CircularNet's vision. Its open-source models, powered by [TensorFlow](https://www.tensorflow.org/) and available on [GitHub](https://github.com/tensorflow/models/tree/master/official/projects/waste_identification_ml), are free to use, customizable, and can help bring analytics to new markets while minimizing cost.
+
+This guide offers step-by-step instructions for setting up and integrating CircularNet, accommodating various deployment options. It describes different deployment preferences so you can install CircularNet according to your needs.
+
+### Get started with CircularNet
+
+Start exploring CircularNet by reviewing the following documentation:  
+
+1. Discover [the benefits, features, components, and use cases](./discover-cn/) of CircularNet.
+1. Choose between [the different deployment options](./solutions/) to install CircularNet models.
+1. Learn about [the recommendations for installing the camera](./system-req/choose-camera/) you require to capture images.
+1. Follow a step-by-step solution example to [deploy CircularNet](./deploy-cn/) and [prepare your captured images](./analyze-data/) for analysis and object tracking.
+1. Learn how to connect your data with [a dashboard for visualization and reporting](./view-data/).

official/projects/waste_identification_ml/circularnet-docs/content/analyze-data/_index.md

@@ -0,0 +1,17 @@
++++
+title = 'Prepare and analyze images'
+date = 2024-07-30T22:02:33Z
+draft = true
+weight = 16
++++
+CircularNet provides two image-analysis models. The first detects _material types_, and the second detects _material forms_. These models utilize a Mask R-CNN algorithm for image training and implement ResNet or MobileNet as the convolutional neural networks for image classification tasks.
+
+The models are loaded sequentially to achieve accurate predictions. When working with images, each image undergoes preprocessing before the models use them for prediction. In the case of video files, the video is split into individual frames at a given frame rate. These frames are then processed in the same sequential manner as images.
+
+The predictions from the two models result in two distinct outputs, which are then post-processed and combined into a single comprehensive output. This output includes critical information such as the number of detected objects, their bounding boxes, class names, class IDs, and masks for each object. Further computer vision techniques extract multiple properties of each object, including color detection. These properties facilitate object tracking and help eliminate duplicate object counts.
+
+If you [deploy the server](../deploy-cn/start-server) on Google Cloud, you can automate the entire image analysis workflow within your VM instance. Integration with BigQuery tables, storage buckets, and dashboards allows for seamless data flow and real-time updates. A [prediction pipeline](./learn-about-pipeline) for Google Cloud pushes the data directly to storage buckets and BigQuery tables, which you can connect to the dashboard for [visualization and analysis](../view-data/).
+
+On the other hand, direct data transfer to the cloud for edge device implementations needs a client-side configuration. A [prediction pipeline](./learn-about-pipeline) for devices lets you load models sequentially and store image analysis results locally.
+
+This section describes how to apply the two specialized CircularNet models using a prediction pipeline on the client side to prepare and analyze the images you capture. 

official/projects/waste_identification_ml/circularnet-docs/content/analyze-data/learn-about-pipeline.md

@@ -0,0 +1,19 @@
++++
+title = 'Learn about the prediction pipeline'
+date = 2024-07-30T22:04:43Z
+draft = true
+weight = 16
++++
+CircularNet offers prediction pipelines for processing, analyzing, and performing object recognition on video or image files. These pipelines facilitate systematic and automated video and image analysis using the Mask R-CNN algorithm and additional object detection and feature extraction processes.
+
+You can run a prediction pipeline from a script to automatically apply the two specialized models that analyze images or video frames. A prediction pipeline operates through the following series of actions in a specific order to ensure reliable and consistent results:
+
+1. Organize your videos or images chronologically according to their creation time or another time-related metadata.
+1. Import files one at a time. If your files are videos, the pipeline decomposes them into individual frames and runs two Mask R-CNN models for pixel-level instance segmentation on each frame.
+1. Implement a color detection algorithm to identify and categorize the colors of the detected objects within the frames or images.
+1. Extract and record features from the detected objects, facilitating analysis and machine learning applications.
+1. Output prediction results of each frame or image with overlaid masks and identification of detected objects.
+
+After processing all frames of a single video, the pipeline implements an object-tracking algorithm to identify and eliminate duplicate occurrences of objects across sequential frames, enhancing the accuracy of object detection and analysis.
+
+Moreover, [applying a prediction pipeline in Google Cloud](./prediction-pipeline-in-cloud) automatically uploads raw images and prediction results into [BigQuery](https://cloud.google.com/bigquery) tables. This seamless integration allows you to combine [visualization dashboards with analytical reports](../view-data/) effortlessly.

official/projects/waste_identification_ml/circularnet-docs/content/analyze-data/prediction-pipeline-in-cloud.md

@@ -0,0 +1,139 @@
++++
+title = 'Apply the prediction pipeline in Google Cloud'
+date = 2024-07-30T22:06:27Z
+draft = true
+weight = 17
++++
+Apart from applying the prediction models to analyze images, the script that runs [the prediction pipeline](./learn-about-pipeline) on Google Cloud ingests data into [BigQuery](https://cloud.google.com/bigquery) to store all the image analysis details.
+
+After [setting up a server](../deploy-cn/start-server) in a Google Cloud account, you can start recording videos of objects passing on your conveyor belt to gather data for analysis. The next step is transferring those video or image files to a [Cloud Storage](https://cloud.google.com/storage) bucket, where the prediction pipeline processes the images.
+
+The results of each video or image prediction are stored and appended to two BigQuery tables, ensuring efficient data management. After systematically processing each file in the Cloud Storage bucket, the pipeline exports and stores the results in another bucket for further use and analysis.
+
+This page explains how to create and manage Cloud Storage buckets on Google Cloud for the videos you record, run the prediction pipeline to apply the models, and process images for further analysis. You can then connect a visualization dashboard to the BigQuery tables to [display results as charts and reports](../view-data/).<br/><br/>
+
+{{< table_of_contents >}}
+
+---
+
+## Store videos in Cloud Storage buckets
+
+To effectively manage and process the videos or images recorded by the [machine vision camera](../system-req/choose-camera/) capturing objects on the conveyor belt, you need the following storage devices:
+
+-  **Local disk storage**: Temporarily cache or store files locally from the camera.
+-  **Cloud Storage input bucket**: Store your recorded videos or images. You can automate uploads using a storage transfer agent.
+-  **Cloud Storage output bucket**: Store the prediction results.
+
+By using these storage options, you ensure an organized, automated, and efficient workflow for managing and processing your recorded videos or images and prediction results.
+
+### Create the Cloud Storage input and output buckets
+
+[Create a Cloud Storage bucket](https://cloud.google.com/storage/docs/creating-buckets) in your Google Cloud account that serves as the _input_ repository for the recorded videos. This bucket stores all the videos or images captured by the machine vision camera and centralizes the storage of your recorded files, making them easily accessible for further processing.
+
+Before uploading to the input bucket, store the files temporarily on your local disk in a predetermined location. This temporary storage ensures you have a local backup and can manage uploads efficiently.
+
+To streamline the upload process to the input bucket, you can schedule a storage job to run at your expected frequency. This automated job takes care of transferring videos or images from the local disk to the Cloud Storage input bucket, relieving you of the manual task. Additionally, implementing a storage transfer agent further facilitates the transfer process, ensuring reliable and efficient uploads. For more information, see [Manage transfer agents](https://cloud.google.com/storage-transfer/docs/managing-on-prem-agents). Otherwise, manually upload your files stored or cached in your local storage to the input bucket before [running the prediction pipeline](#run-the-prediction-pipeline). For more information about manual uploads, see [Upload objects from a file system](https://cloud.google.com/storage/docs/uploading-objects).
+
+In addition to the input bucket, create another Cloud Storage _output_ bucket to store the results generated by your prediction pipeline. This output bucket serves as a well-organized repository for the processed data, such as predictions and analysis results, ensuring that the output from your models is easily accessible and you can manage your pipeline's output.
+
+## Grant required permissions
+
+The [service account](https://cloud.google.com/compute/docs/access/service-accounts#default_service_account) from your NVIDIA T4 GPU virtual machine (VM) instance must have access to Cloud Storage buckets and BigQuery tables to let the VM instance run the prediction pipeline and upload the results to BigQuery.
+
+The VM you created when [deploying CircularNet](../deploy-cn/) has a service account that must act as a _principal_ with the following two roles:  
+
+-  Grant the Storage Admin role to access your Cloud Storage input and output buckets. [Add the VM service account as a principal to a bucket-level policy](https://cloud.google.com/storage/docs/access-control/using-iam-permissions#bucket-add) for both buckets.
+-  Grant the BigQuery Admin role to access your BigQuery tables. [Manage roles of the VM service account using the Google Cloud console](https://cloud.google.com/iam/docs/manage-access-service-accounts).
+
+## Run the prediction pipeline
+
+Follow these steps to run the prediction pipeline and process your input files on Google Cloud:
+
+1. [Open the Google Cloud console](https://cloud.google.com/cloud-console).
+1. [Create the Cloud Storage input and output buckets](#create-the-cloud-storage-input-and-output-buckets).
+1. Upload the images or videos your machine vision camera captured to the Cloud Storage input bucket.
+1. [Grant  the required permissions to the VM service account](#grant-required-permissions).
+1. From the **Navigation menu** on the Google Cloud console, select **Compute Engine** > **VM instances**.
+1. On the **VM instances** page, find the VM instance you created with the NVIDIA T4 GPU when [deploying CircularNet](../deploy-cn/).
+1. If you stopped your VM instance, restart it by clicking **More actions** > **Start / Resume** in the row of the instance that you want to restart.  
+
+    **Note:** Stop the instance after you finish using it to avoid unnecessary expenses.  
+
+1. Click **SSH** in the row of the instance that you want to connect to. The **SSH-in-Browser** tool opens. For more information, see [Connect to VMs](https://cloud.google.com/compute/docs/connect/standard-ssh#connect_to_vms).
+1. On the **SSH-in-browser** window, [start the server](../deploy-cn/start-server).
+1. Display the names of the models you loaded to the Triton inference server:
+
+    ```
+    cat triton_server.sh
+    ```
+
+	The first lines of the output show the names of the loaded models in square brackets. You can call any of these models when running the prediction pipeline.  
+	  
+	**Important:** Run the previous command in the `server` folder, which contains the `triton_server.sh` script.  
+
+1. Exit the `server` folder and open the client folder in the `prediction_pipeline` directory:
+
+    ```
+    cd ..
+    cd client/
+    ```
+
+	This folder contains the `pipeline_images.py` and `pipeline_videos.py` Python files that store the complete prediction pipelines for input images or videos, respectively. The `run_gcp_images.sh` and `run_gcp_videos.sh` scripts run these Python files automatically.  
+
+1. If you have to modify the scripts to provide your specific paths and values for the prediction pipeline, edit the corresponding parameter values on the script. The following example modifies the video pipeline script:
+
+    ```
+    vim run_gcp_videos.sh
+    ```
+
+	The Vim editor displays the following parameters:
+
+    ```
+    --input_directoy=<path-to-input-bucket>
+    --output_directory=<path-to-output-bucket>
+    --fps=<frames-per-second>
+    --height=<height>
+    --width=<width>
+    --material_model=<material-model>
+    --material_form_model=<material-form-model>
+    --score=<score>
+    --search_range=<search-range>
+    --memory=<memory>
+    --project_id=<project-id>
+    --dataset_id=<dataset-id>
+    --table_id=<table-id>
+    ```
+
+    Replace the following:
+
+    -  `<path-to-input-bucket>`: The path to [the Cloud Storage input bucket you created](#create-the-cloud-storage-input-and-output-buckets), for example `gs://my-input-bucket/`. 
+    -  `<path-to-output-bucket>`: The path to [the Cloud Storage output bucket you created](#create-the-cloud-storage-input-and-output-buckets), for example `gs://my-output-bucket/`. 
+    -  `<frames-per-second>`: The rate at which you want to capture images from videos to split videos into frames, for example, 15.
+    -  `<height>`: The height in pixels of the image or video frames that the model expects for prediction, for example, 512.
+    -  `<width>`: The width in pixels of the image or video frames that the model expects for prediction, for example, 1024.
+    -  `<material-model>`: The name of the material model in the Triton inference server that you want to call, for example, `material_resnet_v2_512_1024`.
+    -  `<material-form-model>`: The name of the material form model in the Triton inference server that you want to call, for example, `material_form_resnet_v2_512_1024`.
+    -  `<score>`: The threshold for model prediction, for example, 0.40.
+    -  `<search-range>`: The pixels up to which you want to track an object for object tracking in consecutive frames, for example, 100.
+    -  `<memory>`: The frames up to which you want to track an object, for example, 20.
+    -  `<project-id>`: The ID of your Google Cloud project, for example, `my-project`.
+    -  `<dataset-id>`: The ID that you want to assign to a BigQuery dataset to store prediction results, for example, `circularnet_dataset`.
+    -  `<table-id>`: The ID that you want to assign to a BigQuery table to store prediction results, for example, `circularnet_table`. If the table already exists in your Google Cloud project, the pipeline appends results to that table.
+
+    **Note:** If your input files are not videos but images, replace `run_gcp_videos.sh` on the command with `run_gcp_images.sh` and remove the `--fps=<frames-per-second>` parameter.
+
+    Save changes and exit the Vim editor. To do this, press the **Esc** key, type `:wq`, and then press **Enter**.  
+
+1. Run the prediction pipeline:
+
+    ```
+    bash run_gcp_videos.sh
+    ```
+
+    **Note:** If your input files are not videos but images, replace `run_gcp_videos.sh` on the command with `run_gcp_images.sh`.
+
+The script also creates a `logs` folder inside the `client` folder that saves the logs with the troubleshooting results and records from the models.
+
+You have finished running the prediction pipeline and applying the prediction models to your files for further analysis. You can find the image results with the applied masks in your output bucket. You can also open the generated [BigQuery](https://cloud.google.com/bigquery) table to see the model analytics results and [preview table data](https://cloud.google.com/bigquery/docs/quickstarts/load-data-console#preview_table_data). To create new results, repeat the steps in this section every time you modify the files in your input bucket.
+
+**Important:** If you rerun the prediction pipeline on the same video or image file, you must delete the results created the first time you ran the script from the output bucket to avoid conflicting issues. [Manage the lifecycle of objects in your Cloud Storage buckets](https://cloud.google.com/storage/docs/lifecycle) to help manage costs.