-
Notifications
You must be signed in to change notification settings - Fork 0
/
evaluation.html
248 lines (220 loc) · 15.3 KB
/
evaluation.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="description" content="">
<meta name="author" content="">
<title>Sapient - CCTV Analysis AI</title>
<!-- Bootstrap core CSS -->
<link href="vendor/bootstrap/css/bootstrap.min.css" rel="stylesheet">
<!-- Custom fonts for this template -->
<link href="vendor/fontawesome-free/css/all.min.css" rel="stylesheet">
<link href="https://fonts.googleapis.com/css?family=Varela+Round" rel="stylesheet">
<link href="https://fonts.googleapis.com/css?family=Nunito:200,200i,300,300i,400,400i,600,600i,700,700i,800,800i,900,900i" rel="stylesheet">
<!-- Custom styles for this template -->
<link href="css/grayscale.min.css" rel="stylesheet">
<link href="css/custom.css" rel="stylesheet">
</head>
<body id="page-top">
<!-- Navigation -->
<nav class="navbar navbar-expand-lg navbar-light fixed-top" id="mainNav">
<div class="container">
<a class="navbar-brand js-scroll-trigger" href="index.html">SAPIENT</a>
<button class="navbar-toggler navbar-toggler-right" type="button" data-toggle="collapse" data-target="#navbarResponsive" aria-controls="navbarResponsive" aria-expanded="false" aria-label="Toggle navigation">
Menu
<i class="fas fa-bars"></i>
</button>
<div class="collapse navbar-collapse" id="navbarResponsive">
<ul class="navbar-nav ml-auto">
<li class="nav-item">
<a class="nav-link js-scroll-trigger" href="requirements.html">Requirements</a>
</li>
<li class="nav-item">
<a class="nav-link js-scroll-trigger" href="research.html">Research</a>
</li>
<li class="nav-item">
<a class="nav-link js-scroll-trigger" href="hci.html">HCI</a>
</li>
<li class="nav-item">
<a class="nav-link js-scroll-trigger" href="design.html">Design</a>
</li>
<li class="nav-item">
<a class="nav-link js-scroll-trigger" href="testing.html">Testing</a>
</li>
<li class="nav-item">
<a class="nav-link js-scroll-trigger" href="evaluation.html">Evaluation</a>
</li>
<li class="nav-item">
<a class="nav-link js-scroll-trigger" href="management.html">Management</a>
</li>
</ul>
</div>
</div>
</nav>
<!-- Header -->
<header class="masthead">
<div class="container d-flex h-100 align-items-center">
<div class="mx-auto text-center">
<h1 class="mx-auto my-0 text-uppercase">Evaluation</h1>
<h2 class="text-white-50 mx-auto mt-2 mb-5">Achievements, Critical Evaluation and Future Work</h2>
<a href="#summary_of_achievements" class="btn btn-primary js-scroll-trigger">Show me more</a>
</div>
</div>
</header>
<div class="container-fluid bg-light">
<div class="row">
<!--Sidebar menu-->
<div class="col-sm-2">
<br>
<br>
<div id="sidebar" class="sidebar">
<nav class="navbar navbar-shrink">
<ul class="navbar-nav">
<li class="nav-item">
<a class="nav-link js-scroll-trigger" href="#summary_of_achievements">Summary Of Achievements</a>
</li>
<li class="nav-item">
<a class="nav-link js-scroll-trigger" href="#critical_evaluation">Critical Evaluation</a>
</li>
<li class="nav-item">
<a class="nav-link js-scroll-trigger" href="#future_work">Future Work</a>
</li>
</ul>
</nav>
</div>
</div>
<div class="col-sm-10">
<!-- Black Spot Beneath Header Section -->
<section id="summary_of_achievements" class="projects-section bg-light">
<div class="container">
<div class="col-lg-12 text-center">
<h1 class="section-heading text-uppercase">Summary of achievements</h1>
<p class="line"></p>
<h2 class="section-heading text-uppercase"><u>Achievement Table</u></h2>
<img src="img/achievement.png" class="achievements_table">
<h2 class="section-heading text-uppercase"><u>Bug list</u></h2>
<img src="img/buglist.png" class="bug_list">
<h2 class="section-heading text-uppercase"><u>Individual Contribution Table</u></h2>
<img src="img/contribution.png" class="contribution_table">
</div>
</div>
</section>
<section id="critical_evaluation" class="projects-section bg-light">
<div class="container">
<div class="col-lg-12 text-center">
<h1 class="section-heading text-uppercase">Critical Evaluation</h1>
<p class="line">
This section is about the critical points of our project, including User Experience, functionality, stability and efficiency.
We will discuss all about the details of our project in that section for future development and anyone have interest in.
</p>
<h2 class="section-heading text-uppercase"><u>User Experience</u></h2>
<p class="text-black-100" style="text-align: justify;">
The user experience is quite good since we have a really nice design of the web application, including the UI, the page layout
and the color scheme. However the speed of the posture recognition is too slow which makes people misunderstand it is working not
shut down. We should including a notification said that 'please go and grab a cofffee then come back to see th results'.
</p>
<p class="text-black-100" style="text-align: justify;">
Also, for the item recognition, the notation of the items detected by the system flash quickly which makes hard to catch up with.
The problem is that the item recognition cannot work for every frames, so it flash once by five or more frames going through. That is a
problem of the API, it cannot read video but only images.
</p>
<p class="text-black-100" style="text-align: justify;">
These user experiecne feed back is important to our project. We will fix it if we have
time in the future.
</p>
<h2 class="section-heading text-uppercase"><u>Functionality</u></h2>
<p class="text-black-100" style="text-align: justify;">
There are three recognitions has been implemented with our project. They are posture recognition, item recognition and face recognition. Posture recognition
detecting people's posture including sitting, standing and lying down. Face recognition recognise the people within the frames. Item recognition
outline the top5 the most significant items within the frame with name and confidence rate.
</p>
<p class="text-black-100" style="text-align: justify;">
The input form can be video uploading by azure virtual system, or live-stream input through web camera by the local flask web server.
</p>
<h2 class="section-heading text-uppercase"><u>Stability</u></h2>
<p class="text-black-100" style="text-align: justify;">
If the uploading video is too long, the recognition system will shut down since the web browser cannot wait for few minutes long response,
it is caused by the natural of web server application, they have a maximum responsing time, if the back-end system took too long to return the
response, it will lost connection and ask the user to refresh the page. We have not fond a efficient way to fix that problem, however if we are using the live
stram on local to play the AI recognition, there won't be any problem with the responsing time since it continuously keep connecting and disconnecting, response with one
analysis frame at each time.
</p>
<h2 class="section-heading text-uppercase"><u>Efficiency</u></h2>
<p class="text-black-100" style="text-align: justify;">
We choose a ten-second video to test for the item and face recognition. There are two people including in the video with one person has a small knives held by hand. We found the face has been nicely detected three time per second accurately within the range of 3 meters and the deadly (small knives) never been recognised when it is 5 meters away from the camera. It shows that how the AI CCTV far away from the real-world application at the moment.
</p>
<br>
<p class="text-black-100" style="text-align: justify;"></p>
<h2 class="section-heading text-uppercase"><u>Compatibility</u></h2>
<br>
<p class="text-black-100" style="text-align: justify;">At the moment, only Edge and FireFox are able to access the webcam for the webapp. In order to improve this compatibility, it can be solved by having an DNS for this webapp and get a SSL certificate as well. So that the website will be accessed via HTTPS, so that all the web-browser we tested can access the webcam due to the restriction from the JS library.</p>
<h2 class="section-heading text-uppercase"><u>Maintainability</u></h2>
<br>
<p class="text-black-100" style="text-align: justify;">From our application, there are few components we have to look after in order to maintain the system.
<br>
<br>
Firstly, Google cloud vision API requires money to run, it is free for the first 1000 image analysis, after that it requires you to pay $1.5 for 1001 to 5,000,000 units, more details are in <i>cloud.google.com/vision/pricing</i>.
<br>
<br>
Secondly, The AzureVM running NC6 requires around £0.9 per hour, and we have to make sure that we have enough budgets in our Azure account It is recommended that you start the machine when you need it and stop the machine when you dont need it.
<br>
<br>
</p>
<h2 class="section-heading text-uppercase"><u>Project Management</u></h2>
<br>
<p class="text-black-100" style="text-align: justify;">At the start, we splited the tasks and roles for this project. Due to the main objective for this project is to develop the AI of the CCTV, we decided to let everyone get involved with the AI part. Therefore, We decided that Andreas to be working on the frontend, posture sample gathering and communication with the client, Shun to be working on the Face and Item recognition and Shiko to work on the deployment, system integration and Posture Recognition.
<br>
<br>
We are happy with our work and the progress of our work, comparing with other group, we took more times than other group to combine backend and frontend together. This is due to the complexity and the difficulty of our project. In order to test and finish our application, we had to work with server, image recognition and the pipeline between the frontend and the backend and we have to let each member to know each part of the application inside out.
<br>
<br>
The difficulty of the management part of this project is managing the time and the workload of integrating the frontend and the backend of the application while you are configuring the image recognition. The image recognition that we built is not something like API that we can use it straight away, we had to create the pipeline to go through image data extraction, datasets and model creation and improve the accuracy on the image recognition. We solved this issue by putting vast majority of time and testing each component to speed up the development progress.
</p>
</div>
</div>
</section>
<section id="future_work" class="projects-section bg-light">
<div class="container">
<div class="col-lg-12 text-center">
<h1 class="section-heading text-uppercase">Future Work</h1>
<p class="line"></p>
<h2 class="section-heading text-uppercase"><u>Web Application</u></h2>
<h2 align="left"><u>Model Customization</u></h2>
<p class="text-black-100" style="text-align: justify;">Further develop the current web application in Azure which can allow the user to input the images sample and then create the model using Multi Layer Perception classifier in order to achieve machine learning by using their own samples. Finally, the user then can select the model they generated to process the image recognition. In the current stage, since the item recognition is using Google Vision API, this functionality is only capable for face and posture recognition. The ultimate goal for this part is having login authentication features to allow multiple users to upload the samples and train the model together.</p>
<br>
<h2 align="left"><u>Live Video</u></h2>
<p class="text-black-100" style="text-align: justify;">Flask, Django is not designed for long time connection, and it is the same for the most other python framework we have to find a way to connect the web camera and system by hardware but not network to do the live stream AI Recognition.</p>
<br>
<br>
<h2 class="section-heading text-uppercase"><u>Image Recognition</u></h2>
<h2 align="left"><u>Posture Recognition</u></h2>
<p class="text-black-100" style="text-align: justify;">Firstly, Training new classifiers with sufficient data to extend the posture recognition capabilities. Secondly, adding new human posture to it. Secondly, increasing the precision on the existing posture that can be recognised by the machine. Finally, it would be great to activate a pop-up notification functionality when there is a fighting behaviour.</p>
<br>
<h2 align="left"><u>Face Recognition</u></h2>
<p class="text-black-100" style="text-align: justify;">When the OpenCV.js is open to self-implement faces library, we will replacing the back-end with the JavaScript commands</p>
<br>
<h2 align="left"><u>Item Recognition</u></h2>
<p class="text-black-100" style="text-align: justify;">For the moment, it can only detect the top5 most significant items within the frame. It would be great to tweak the item recognition into instead of recognising all the items in a frame, finding dangerous items such as gun, knives would meet the expectation from Camden Council.</p>
</div>
</div>
</section>
</div>
</div>
</div>
<!-- Footer -->
<footer class="bg-black small text-center text-white-50">
<div class="container">
Copyright © Sapient Platform 2018
</div>
</footer>
<!-- Bootstrap core JavaScript -->
<script src="vendor/jquery/jquery.min.js"></script>
<script src="vendor/bootstrap/js/bootstrap.bundle.min.js"></script>
<!-- Plugin JavaScript -->
<script src="vendor/jquery-easing/jquery.easing.min.js"></script>
<!-- Custom scripts for this template -->
<script src="js/grayscale.min.js"></script>
<script src="js/sidebar.js"></script>
</body>
</html>