Issue157 - YouTube Data API / Captions

.travis.yml

@@ -9,4 +9,5 @@ php:
   - '5.4'
   - '5.5'
   - '5.6'
-script: ./vendor/phpunit/phpunit/phpunit
+script: 
+  - './vendor/phpunit/phpunit/phpunit --testsuite "UDOIT"'

Dockerfile

@@ -0,0 +1,6 @@
+# Get php 5.6 
+FROM php:5.6-alpine
+
+RUN apk update && apk add git
+
+WORKDIR /var/www

HEROKU.md

@@ -3,6 +3,57 @@ Use the button below to quick start your deploy.
 
 [![Deploy](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy)
 
+## Installation Instructions
+Installing UDOIT using the Heroku button is very easy, but still requires some setup.  If you prefer to watch a video demonstrating the process step-by-step, watch the [UDOIT Installation CanvasLIVE video](https://www.youtube.com/watch?v=g1LgnErkvsA).
+
+Below are the written directions if you prefer to follow along that way.
+
+### Step 1:  Create a Google/YouTube API key
+
+See the [Google/YouTube API instructions](README.md#googleyoutube-api-key) in the README.
+
+### Step 2:  Setting up Heroku
+After clicking the Heroku button above:
+
+1. Create an account (if you don't have one already).
+2. Give the app a name.
+3. Fill out the `OAUTH2_ID` and `OAUTH2_KEY` fields with dummy data.  (We'll fix it later.)
+4. Fill out the `OAUTH2_URI` field with `https://yourapp.herokuapp.com/oauth2response.php`. (Replace 'yourapp' with the name you gave in step 2.)
+5. Copy and paste your Google/YouTube API key.
+5. Click the Deploy button and wait for the process to complete.
+
+### Step 3:  Request a Developer Key
+UDOIT uses Oauth2 to take actions on behalf of the user, so you'll need to ask your Canvas administrator to generate a Developer Key for you.  (If you are an admin, go to your institution's account administration page in Canvas and click on 'Developer Keys'.)  Here is the information you need to provide them:
+
+* ***Key Name:*** Probably ***UDOIT*** or ***UDOIT Test*** for your test instance
+* ***Owner Email:*** The email address of whoever is responsible for UDOIT at your institution
+* ***Redirect URI:*** This is the URI of the `oauth2response.php` file in the UDOIT directory.
+ * This should be `https://yourapp.herokuapp.com/oauth2response.php`. (Replace 'yourapp' with the name of your UDOIT instance on Heroku.)
+* ***Icon URL:*** The URL of the UDOIT icon.  This is `https://yourapp.herokuapp.com/assets/img/udoit_icon.png`.  (Replace ***yourapp*** with the name of your UDOIT instance on Heroku.)
+
+### Step 4:  Add your Developer Key to UDOIT
+1. In Heroku, click the 'Manage App' button for your install of UDOIT.
+2. Go to the 'Settings' tab.
+3. Copy and paste the following values from the Developer Key:
+ * ID into ***OAUTH2_ID***
+ * Secret into ***OAUTH2_KEY***
+4. Verify that your ***OAUTH2_URI*** is correct. (See above.)
+
+### Step 5:  Install the UDOIT LTI
+
+1. You can install UDOIT at the sub-account level or the course level.  Either way, start by going to the **settings** area.
+2. Click the **Apps** tab.
+3. Click the **View App Configurations** button.
+4. Click the **Add App** button.
+5. Under **Configuration Type**, choose **By URL**.
+6. In the **Name** field, enter `UDOIT`.
+7. In the **Consumer Key** field, copy the value from OAUTH2_ID
+8. In the **Shared Secret** field, copy the value from OAUTH2_KEY
+9. In the **Config URL** field, paste value from OAUTH2_URI.
+10. Finish by clicking **Submit**.
+
+UDOIT should now be available in the course navigation menu.
+
 # Manual Deploy on Heroku
 
 You can use our configuration to launch a new Heroku app using [Heroku's app-setups api](https://devcenter.heroku.com/articles/setting-up-apps-using-the-heroku-platform-api).
@@ -34,7 +85,7 @@ These variables can be set in the curl post above.  You can also set them later
 * `SHARED_SECRET` - LTI secret entered when adding UDOIT LTI to Canvas
 * `OAUTH2_ID` - from the developer api key created by your admin
 * `OAUTH2_KEY` - from the developer api key created by your admin
-* `OAUTH2_URI` - full url to your oauth2responce.php - EX: `https://your.herokuapp.com/oauth2response.php`
+* `OAUTH2_URI` - full url to your oauth2response.php - EX: `https://your.herokuapp.com/oauth2response.php`
 * `GOOGLE_API_KEY` - add a google api key for youtube video support
 * `USE_HEROKU_CONFIG` - set to `true` to enable the Heroku configuration
 

README.md

@@ -98,34 +98,55 @@ $ php lib/db_create_tables.php
 The table schema can be found in [lib/db_create_tables.php](lib/db_create_tables.php)
 
 ## Configuration and Setup
-If you didn't already make `config/localConfig.php` when you set up the database do it now.
+If you didn't already make `config/localConfig.php` when you set up the database, do it now.
 
 ### Canvas API
 Please refer to the [Canvas API Policy](http://www.canvaslms.com/policies/api-policy) before using this application, as it makes heavy use of the Canvas API.
 
+### LTI Security
+UDOIT uses the security processes built into the LTI specification to ensure that users are only accessing UDOIT from within your instance of Canvas.  There are two values that need to be set in order for this security process to work.  These values should be different from each other.  You will use them again when you are installing the LTI in Canvas.
+
 Edit `config/localConfig.php`:
 
-* `$consumer_key`: A consumer key you make up.  Used when installing the LTI in Canvas.
-* `$shared_secret`: The shared secret you make up.  Used when installing the LTI in Canvas.
+* `$consumer_key`: A value you make up.
+* `$shared_secret`: The value you make up.
 
 ### Canvas Oauth2
-UDOIT uses Oauth2 to take actions on behalf of the user, you'll need to [sign up for a developer key](https://docs.google.com/forms/d/1C5vOpWHAAl-cltj2944-NM0w16AiCvKQFJae3euwwM8/viewform)
+UDOIT uses Oauth2 to take actions on behalf of the user, so you'll need to ask your Canvas administrator to generate a Developer Key for you.  Here is the information you need to provide them:
 
-Edit `config/localConfig.php`:
+* ***Key Name:*** Probably ***UDOIT*** or ***UDOIT Test*** for your test instance
+* ***Owner Email:*** The email address of whoever is responsible for UDOIT at your institution
+* ***Redirect URI:*** This is the URI of the `oauth2response.php` file in the UDOIT directory.
+ * If you did a normal install into the web root of your server, it would be `https://www.example.com/public/oauth2response.php`. (Replace 'www.example.com' with the url of your UDOIT server.)
+* ***Icon URL:*** The URL of the UDOIT icon.  This is `https://www.example.com/public/assets/img/udoit_icon.png`.  (Replace 'www.example.com' with the url of your UDOIT server.)
+
+After you receive your Developer Key from your Canvas admin, edit the following variables in `config/localConfig.php`:
+
+* `$oauth2_id`: The Client_ID yoru Canvas admin gives you
+* `$oauth2_key`: The Secret your Canvas admin gives you
+* `$oauth2_uri`: The Redirect URI you provided to your Canvas admin
+
+### Google/YouTube API Key
+In order for UDOIT to scan YouTube videos for closed captioning, you will need to create a YouTube Data API key.  Follow the instructions below:
 
-* `$oauth2_id`: The Client_ID Instructure gives you
-* `$oauth2_key`: The Secret Instructure gives you
-* `$oauth2_uri`: The "Oauth2 Redirect URI" you provided Instructure.  This is the URI of the `auth2response.php` file in the UDOIT directory.
+1. Go to the [Google Developer Console](https://console.developers.google.com).
+2. Create a project.
+3. Enable ***YouTube Data API V3***
+4. Create an ***API key*** credential.
 
 ### Installing the LTI in Canvas
 Log into Canvas to add UDOIT:
 
-1. Under **Configuration Type**, choose **By URL**.
-2. In the **Name** field, enter `UDOIT`.
-3. In the **Consumer Key** field, copy the value from `$consumer_key` from `config/localConfig.php`
-4. In the **Shared Secret** field, copy the value from `$shared_secret` from `config/localConfig.php`
-5. In the **Config URL** field, paste the **FULL URL** that points to `udoit.xml.php`. **See** LTI Config URL Notes.
-6. Finish by clicking **Submit**.
+1. You can install UDOIT at the sub-account level or the course level.  Either way, start by going to the **settings** area.
+2. Click the **Apps** tab.
+3. Click the **View App Configurations** button.
+4. Click the **Add App** button.
+5. Under **Configuration Type**, choose **By URL**.
+6. In the **Name** field, enter `UDOIT`.
+7. In the **Consumer Key** field, copy the value from `$consumer_key` from `config/localConfig.php`
+8. In the **Shared Secret** field, copy the value from `$shared_secret` from `config/localConfig.php`
+9. In the **Config URL** field, paste the **FULL URL** that points to `udoit.xml.php`. **See** LTI Config URL Notes.
+10. Finish by clicking **Submit**.
 
 #### LTI Config URL Notes
 The URL of your UDOIT LTI config depends on your webserver install.  The file is located the `public` directory. The examples below should give you are some possible values:
@@ -156,6 +177,12 @@ We use phpunit to run unit tests on UDOIT.  To run the tests, type the following
 $ ./vendor/phpunit/phpunit/phpunit
 ```
 
+By default, phpunit will run all tests, including the functional tests that require access to outside APIs.  If you would like to exclude those tests, run this command:
+
+```
+$ ./vendor/phpunit/phpunit/phpunit --exclude-group functional
+```
+
 ## Contributors
 * [Jacob Bates](https://github.com/bagofarms)
 * [Eric Colon](https://github.com/accell)

composer_depend.sh

@@ -0,0 +1,16 @@
+#! /usr/bin/env ash
+#  This script is intended to update our projects composer dependencies.
+#  This also runs phpunit tests.
+
+# exit on failure
+set -e
+
+echo Starting composer_depend.sh script...
+
+php composer.phar self-update
+
+php composer.phar update --no-scripts
+
+vendor/phpunit/phpunit/phpunit
+
+echo Finished composer_depend.sh script...

config/tests.php

@@ -157,7 +157,7 @@
         [
             'name'      => 'videosEmbeddedOrLinkedNeedCaptions',
             'title'     => 'Synchronized <a href="http://webaim.org/techniques/captions/">captions</a> should be provided for prerecorded web-based video',
-            'desc'      => '<p>Captions should be included in the video to provide dialogue to users who are hearing impaired.</p>',
+            'desc'      => '<p>Captions should be included in the video to provide dialogue to users who are hearing impaired.  (Please note that videos that have been removed, deleted, or are Unlisted will also cause this error, and will need to be manually verified.)</p>',
             'resources' => [
                 '<a href="https://support.google.com/youtube/answer/2734796?hl=en">Adding Captions to Youtube</a>',
                 '<a href="http://guides.instructure.com/m/4152/l/98632-how-do-i-create-captions-for-new-or-uploaded-videos-in-canvas">Creating Captions for Video Uploaded to Canvas</a>',

docker-compose.yml

@@ -0,0 +1,5 @@
+web:
+  build: .
+  volumes:
+    - .:/var/www
+  command: ./composer_depend.sh

lib/quail/quail/common/services/media/youtube.php

@@ -28,7 +28,7 @@ class youtubeService extends mediaService
 	/**
 	*	@var string The service point to request caption data from YouTube
 	*/
-	var $search_url = 'https://www.googleapis.com/youtube/v3/videos?part=contentDetails&id=';
+	var $search_url = 'https://www.googleapis.com/youtube/v3/captions?part=snippet&videoId=';
 
 	/**
 	*	Checks to see if a video is missing caption information in YouTube
@@ -37,25 +37,30 @@ class youtubeService extends mediaService
 	*/
 	function captionsMissing($link_url)
 	{
-
-		$url = 'https://www.googleapis.com/youtube/v3/captions?part=id&videoId='; //'https://www.googleapis.com/youtube/v3/videos?part=contentDetails&id=';
+		$url = $this->search_url;
 		$api_key = constant( 'GOOGLE_API_KEY' );
 
 		if( $youtube_id = $this->isYouTubeVideo($link_url) ) {
 			$url = $url.$youtube_id.'&key='.$api_key;
 			$response = Request::get($url)->send();
 
-			if( !$response ) {
+			// If the video was pulled due to copyright violations, is unlisted, or is unavailable, the items array will be empty.
+			// Another error will result in this case
+			if( empty($response->body->items) ) {
 				return true;
 			}
 
-			// If the video was pulled due to copyright violations, the items array will be empty.
-			// TODO:  Make this return a different error, warning the instructor that the video is no longer available
-			return ( empty($response->body->items) )? true : false;
+			foreach ( $response->body->items as $track ) {
+				if ( $track->snippet->trackKind != 'ASR' ) {
+					return false;
+				}
+			}
+
+			return true;
 		}
 
 		return false;
-		
+
 	}
 
 	/**

lib/quail/quail/guidelines/translations/en.txt

@@ -152,7 +152,7 @@
 "embedMustNotHaveEmptyAlt","""Embed"" elements cannot have an empty ""alt"" attribute","<p>All <code>embed</code> elements should have an ""alt"" attribute that is not empty.</p><h4>Example</h4><h5>Wrong</h5><p><code>&lt;embed src=""dog.mov"" alt=""""/&gt;</code></p><h5>Right</h5><p><code>&lt;embed src=""dog.mov"" alt=""A movie featuring a dog dancing a ballet.""/&gt;</code></p>","1"
 "iframeMustNotHaveLongdesc","Inline frames (""iframes"") should not have a ""longdesc"" attribute","<p><code>Iframe</code> elements should not have a ""longdesc"" attribute.</p>","1"
 "iframeIsNotUsed","Iframes need to be checked to make sure content within is accessible","<p><code>Iframe</code> elements should be checked to make sure content within is accessible.</p>","2"
-"videoEmbedChecked","Synchronized captions should be provided for prerecorded web-based video","<p>Captions should be included in the video to provide dialogue to users who are hearing impaired.</p>","2"
+"videoEmbedChecked","Synchronized captions should be provided for prerecorded web-based video","<p>Captions should be included in the video to provide dialogue to users who are hearing impaired.  (Please note that videos that have been removed, deleted, or are Unlisted will also cause this error, and will need to be manually verified.)</p>","2"
 "radioMarkedWithFieldgroupAndLegend","All radio button groups are marked using fieldset and legend elements.","form element content should contain both fieldset and legend elements if there are related radio buttons.","1"
 "selectWithOptionsHasOptgroup","Form select elements should use optgroups for long selections","<p><code>Select</code> form elements with long lists of items should use the <code>optgroup</code> tag to group like selections together.</p>","2"
 "aSuspiciousLinkText","Link text should be descriptive","<p>Links should be descriptive of the content they're linking to, such as 'Class Schedule' rather than 'schedule.html' or 'click here'.</p> ","1"
@@ -235,7 +235,7 @@
 "cssTextHasContrast","Insufficient text color contrast with the background","<p>Text color should be easily viewable and should not be the only indicator of meaning or function. Color balance should have at least a 4.5:1 ratio for small text and 3:1 ratio for large text.</p>","1"
 "cssTextStyleEmphasize","Avoid using color alone for emphasis","<p>When emphasizing text, you may use color with sufficient contrast as long as you also apply some other form of emphasis, such as bold or italics. This ensures that screen reader users are aware of the text's importance.</p>","1"
 "videoProvidesCaptions","All video tags should provide captions","<p>Videos used on online courses are required to have closed captioning. Unfortunately, automatic captioning (such as on YouTube videos) is not accurate enough for educational use. You have three options:</p><ul><li>Contact the creator of the video and request captions be added.</li><li>Create captions yourself using a service like Amara (http://amara.org/).</li><li>Find a different video that has closed captioning.</li></ul>","2"
-"videosEmbeddedOrLinkedNeedCaptions","Synchronized <a href='http://webaim.org/techniques/captions/'>captions</a> should be provided for prerecorded web-based video","<p>Captions should be included in the video to provide dialogue to users who are hearing impaired.</p>","1"
+"videosEmbeddedOrLinkedNeedCaptions","Synchronized <a href='http://webaim.org/techniques/captions/'>captions</a> should be provided for prerecorded web-based video","<p>Captions should be included in the video to provide dialogue to users who are hearing impaired.  (Please note that videos that have been removed, deleted, or are Unlisted will also cause this error, and will need to be manually verified.)</p>","1"
 "documentIsWrittenClearly","The document should be written to the target audience and read clearly","<p>If a document is beyond a 10th grade level, then a summary or other guide should also be provided to guide the user through the content.</p>","2"
 "headersHaveText","Headings should contain text","<p>Sighted and <a href='http://en.wikipedia.org/wiki/Screen_reader'>screen reader</a> users depend on headings to organize the content on the page. Headings should not be empty and should represent an accurate outline of the content</p>","1"
 "labelsAreAssignedToAnInput","All labels should be associated with an input","<p>All <code>label</code> elements should be assigned to an input item, and should have a <em>for</em> attribute which equals the <em>id</em> attribute of a form element.</p>","1"

phpunit.xml

@@ -3,5 +3,8 @@
     <testsuite name="UDOIT">
       <file>tests/UfixitTest.php</file>
     </testsuite>
+    <testsuite>
+    	<file>tests/FunctionalTests.php</file>
+    </testsuite>
   </testsuites>
 </phpunit>

run_tests.sh

@@ -0,0 +1 @@
+docker-compose run --rm web ./composer_depend.sh

tests/FunctionalTests.php

@@ -0,0 +1,41 @@
+<?php
+// TODO: Figure out a way to not load these here.  Tests should be self-contained.
+// That requires rewriting and restructuring the project, so we're compromising for now.
+include_once('lib/quail/quail/common/services/media/youtube.php');
+include_once('config/settings.php');
+
+class FunctionalTests extends PHPUnit_Framework_TestCase
+{
+    // public function setUp () {
+
+    // }
+
+    // public function tearDown () {
+
+    // }
+
+    // private function checkOutputBuffer() {
+    //     $buffer         = ob_get_clean();
+    //     $this->assertEquals('', $buffer);
+    // }
+
+    /* Tests the youtube api call to make sure a video with captions is detected as having captions */
+    public function testYouTubeAPIHasCaptions() {
+        $vid_url = 'https://www.youtube.com/watch?v=zo6aRvf-l_s';
+
+        $yt_service = new youtubeService();
+        $captions_missing = $yt_service->captionsMissing($vid_url);
+
+        $this->assertFalse($captions_missing);
+    }
+
+    /* Tests the youtube api call to make sure a video without captions is detected as not having captions */
+    public function testYouTubeAPINoCaptions() {
+        $vid_url = 'https://www.youtube.com/watch?v=nBH89Y0Xj7c';
+
+        $yt_service = new youtubeService();
+        $captions_missing = $yt_service->captionsMissing($vid_url);
+
+        $this->assertTrue($captions_missing);
+    }
+}