Partially addressed issue CiscoDevNet#917 (HTTPS support for Restconf)

ygorelik · Jul 16, 2020 · aa59850 · aa59850
1 parent e411c71
commit aa59850
Show file tree

Hide file tree

Showing 12 changed files with 250 additions and 36 deletions.
diff --git a/CHANGES.md b/CHANGES.md
@@ -8,6 +8,7 @@
 #### Resolved GitHub issues
   * gNMI set RPC fails when JSON payload is formatted string
   * Enhance generator to indicate how a YANG modules was processed ([#894](https://github.com/CiscoDevNet/ydk-gen/issues/894))
+  * Restconf in YDK should support HTTPS (partialy resolved) ([#917](https://github.com/CiscoDevNet/ydk-gen/issues/917))
   * Go fails to process filters on enum and identity leaves and leaf-lists ([#969](https://github.com/CiscoDevNet/ydk-gen/issues/969))
   * YDK fails create bundle package when package name contains '.' ([#978](https://github.com/CiscoDevNet/ydk-gen/issues/978))
   * python bundle generator must escape 'async' as variable name ([#980](https://github.com/CiscoDevNet/ydk-gen/issues/980))

diff --git a/artifacts/ubuntu/bionic/libydk-0.8.5-1.amd64.deb b/artifacts/ubuntu/bionic/libydk-0.8.5-1.amd64.deb
diff --git a/artifacts/ubuntu/bionic/libydk_gnmi-0.4.0-5.amd64.deb b/artifacts/ubuntu/bionic/libydk_gnmi-0.4.0-5.amd64.deb
diff --git a/sdk/cpp/CHANGES.md b/sdk/cpp/CHANGES.md
@@ -9,6 +9,7 @@
   * Fix JsonSubtreeCodec only serializing up to 4B numbers
   * Fixed bug in extracting module name from namespace
   * gNMI set RPC fails when JSON payload is formatted string
+  * Restconf in YDK should support HTTPS (partialy resolved) ([#917](https://github.com/CiscoDevNet/ydk-gen/issues/917))
   * C++ YList class fails to process key values when its name contains '-' character ([#997](https://github.com/CiscoDevNet/ydk-gen/issues/997))
   * create_datanode in Path API returns incorrect object and sets incorrect value ([#1003](https://github.com/CiscoDevNet/ydk-gen/issues/1003))
   * create_datanode rejects absolute paths with a leading "/" as defined in RFC 7950 ([#1005](https://github.com/CiscoDevNet/ydk-gen/issues/1005))

diff --git a/sdk/cpp/core/docsgen/guides/guides.rst b/sdk/cpp/core/docsgen/guides/guides.rst
@@ -9,6 +9,7 @@ YDK C++ provides a new interface in the form of Path API, which can be used to w
 
     introduction.rst
     howto_opendaylight.rst
+    restconf_https.rst
     howto_path.rst
     howto_rpc.rst
     presence_class.rst

diff --git a/sdk/cpp/core/docsgen/guides/howto_gnmi.rst b/sdk/cpp/core/docsgen/guides/howto_gnmi.rst
@@ -1,13 +1,13 @@
 Using gNMI with YDK
-============================
+===================
 .. contents::
 
 YDK makes it easy to interact with gNMI programmatically using the YANG model APIs.
 
 Applications can be written using the C++ model API in conjunction with a service and a provider.
 
 Writing the app
-----------------
+---------------
 
 In this example, we set some BGP configuration using the Cisco IOS XR model, the :cpp:class:`CRUD (Create/Read/Update/Delete) service<ydk::CrudService>` and the :cpp:class:`gNMI service provider<ydk::gNMIServiceProvider>`. The example in this document is a simplified version of the more complete sample that is available in ``core/samples/bgp_gnmi_create.cpp``. Assuming you have performed the core and cisco-ios-xr bundle installations first, that more complete sample can be run with the below steps:
 
@@ -55,7 +55,7 @@ In our example YDK application, first, let us include the necessary header files
  using namespace Cisco_IOS_XR_ipv4_bgp_datatypes;
 
 gNMI service provider
-------------------------------
+---------------------
 
 The first step in any application is to create a service provider instance. In this case, the gNMI service provider is responsible for mapping between the CRUD service API and the underlying manageability protocol (gNMI).
 
@@ -69,7 +69,7 @@ We first instantiate a :cpp:class:`Repository<ydk::path::Repository>` using the
 
 
 Using the model APIs
-----------------------
+--------------------
 After establishing the connection, we instantiate the entities and set some data. Now, create an :cpp:class:`Cisco IOS XR BGP<ydk::Cisco_IOS_XR_ipv4_bgp_cfg::Bgp>` configuration object and set the attributes
 
 .. code-block:: c++
@@ -100,7 +100,7 @@ After establishing the connection, we instantiate the entities and set some data
  bgp->instance.append(instance);
 
 Invoking the CRUD Service
----------------------------
+-------------------------
 The CRUD service provides methods to create, read, update and delete entities on a device making use of the session provided by a service provider.  In order to use the CRUD service, we need to instantiate the :cpp:class:`CrudService<ydk::CrudService>` class
 
 .. code-block:: c++
@@ -126,7 +126,7 @@ Finally, we invoke the create method of the :cpp:class:`CrudService<ydk::CrudSer
 Note if there were any errors the above API will raise an exception with the base type :cpp:class:`YServiceError<ydk::YServiceError>`
 
 Logging
-----------------------
+-------
 
 YDK uses the `spdlog` logging library. The logging can be enabled as follows by creating a logger called "ydk". For other options like logging the "ydk" log to a file, see the `spdlog reference <https://github.com/gabime/spdlog#usage-example>`_.
 

diff --git a/sdk/cpp/core/docsgen/guides/restconf_https.rst b/sdk/cpp/core/docsgen/guides/restconf_https.rst
@@ -0,0 +1,105 @@
+..
+  # ***************************************************************
+  # YDK - YANG Development Kit 
+  #  Copyright 2020 Yan Gorelik, YDK Solutions. All rights reserved
+  # ***************************************************************
+  # Licensed to the Apache Software Foundation (ASF) under one
+  # or more contributor license agreements.  See the NOTICE file
+  # distributed with this work for additional information
+  # regarding copyright ownership.  The ASF licenses this file
+  # to you under the Apache License, Version 2.0 (the
+  # "License"); you may not use this file except in compliance
+  # with the License.  You may obtain a copy of the License at
+  #
+  #   http:#www.apache.org/licenses/LICENSE-2.0
+  #
+  #  Unless required by applicable law or agreed to in writing,
+  # software distributed under the License is distributed on an
+  # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  # KIND, either express or implied.  See the License for the
+  # specific language governing permissions and limitations
+  # under the License.
+  # ***************************************************************
+
+Using Restconf with HTTPS
+=========================
+
+By default the RestconfServiceProvider initializes to support HTTP non-secure protocol.
+But YDK also provides partial support for HTTPS protocol. Here 'partial' means that YDK is capable communicate over secure protocol,
+provides data encryption, checks Restconf server CA certificate, but the peer and host name verifications are permanently disabled.
+This limitation should be addressed in future YDK releases.
+
+In order to enable HTTPS protocol, the user must obtain and install the Restconf server CA certificate on application server.
+In Ubuntu the installation procedure is as followed:
+
+CA Certificate Installation
+---------------------------
+
+In order to enable HTTPS protocol, the user must obtain and install the Restconf server CA certificate on application server.
+On Ubuntu the installation procedure is as followed:
+
+.. code-block:: sh
+
+    cd /usr/local/share/ca-certificates/
+    sudo mkdir ydk
+    cp ~/myrestconf.crt ydk/
+    # Make sure the permissions are OK (755 for the folder, 644 for the file)
+    sudo update-ca-certificates
+    # In the output of the last command check that the certificate was added
+
+The installation procedure on CentOS-7:
+
+.. code-block:: sh
+
+    sudo cp ~/myrestconf.crt /etc/pki/ca-trust/source/anchors/
+    sudo update-ca-trust
+
+Getting Mac-OSX to trust self-signed SSL Certificates:
+
+ 1. Locate your CA certificate file.
+ 2. Open up Keychain Access.
+ 3. Drag your certificate into Keychain Access.
+ 4. Go into the Certificates section and locate the certificate you just added.
+ 5. Double click on it, enter the trust section and under “When using this certificate” select “Always Trust”.
+
+
+Code Snippet
+------------
+
+In the application the user must explicitly specify HTTPS protocol in the host address.
+The following example shows, how the RestconfServiceProvider is used to read names of all interfaces from secure Restconf server:
+
+.. code-block:: c++
+ :linenos:
+
+ #include <ydk/types.hpp>
+ #include <ydk/restconf_provider.hpp>
+ #include <ydk/crud_service.hpp>
+ #include "ydk/path_api.hpp"
+
+ #include <ydk_openconfig/openconfig_interfaces.hpp>
+
+ using namespace ydk;
+ using namespace openconfig;
+ using namespace std;
+
+ int main(int argc, char* argv[])
+ {
+    auto repo = path::Repository("/home/ygorelik/.ydk/ios-xe-mgmt.cisco.com");
+    RestconfServiceProvider provider(repo,
+                                     "https://myrestconf.server.com",   // add 'https://' prefix to the host name or address
+                                     "admin-user", "admin-password",
+                                     443,     // HTTPS port
+                                     EncodingFormat::JSON);
+    auto interfaces = openconfig_interfaces::Interfaces{};
+
+    CrudService crud{};
+    auto reply = crud.read(provider, interfaces);
+    auto all_interfaces = dynamic_cast<openconfig_interfaces::Interfaces*>(reply.get());
+
+    for (auto intf_entity : all_interfaces->interface.entities())
+    {
+        auto intf = dynamic_cast<openconfig_interfaces::Interfaces::Interface*>(intf_entity.get());
+        cout << intf->name << endl;
+    }
+ }
diff --git a/sdk/cpp/core/src/path/repository.cpp b/sdk/cpp/core/src/path/repository.cpp
@@ -357,7 +357,7 @@ ydk::path::RepositoryPtr::get_new_ly_modules_from_lookup(ly_ctx* ctx,
             }
             else if (k.length() > strlen("http://") && k.substr(0, strlen("http://")) == "http://")
             {
-                YLOG_ERROR("Failed to find namespace '{}' in lookup table", k);
+                YLOG_INFO("Failed to find namespace '{}' in lookup table", k);
                 continue;
             }
             else {

diff --git a/sdk/cpp/core/src/restconf_client.cpp b/sdk/cpp/core/src/restconf_client.cpp
@@ -42,11 +42,11 @@ static bool token_not_found(size_t token);
 
 
 RestconfClient::RestconfClient(const string & address, const string & username, const string & password,
-                        int port, const string & encoding)
-    : curl(NULL), header_options_list(NULL), encoding(encoding)
+                        int port, const string & encoding, uint ssl_options)
+    : curl(NULL), header_options_list(NULL), encoding(encoding), ssl_options(ssl_options)
 {
-    initialize(address, username, password, port);
     protocol = (address.find("https://") == 0) ? "HTTPS" : "HTTP";
+    initialize(address, username, password, port);
     YLOG_INFO("Ready to communicate with {} using {}", base_url, protocol);
 }
 
@@ -150,31 +150,32 @@ void RestconfClient::initialize_curl(const string & username, const string & pas
     if (protocol == "HTTPS")
     {
         curl_easy_setopt(curl, CURLOPT_USE_SSL, CURLUSESSL_TRY);
-//#ifdef SKIP_PEER_VERIFICATION
-        /*
-         * If you want to connect to a site who isn't using a certificate that is
-         * signed by one of the certs in the CA bundle you have, you can skip the
-         * verification of the server's certificate. This makes the connection
-         * A LOT LESS SECURE.
-         *
-         * If you have a CA cert for the server stored someplace else than in the
-         * default bundle, then the CURLOPT_CAPATH option might come handy for
-         * you.
-         */
-        curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
-//#else
-        // curl_easy_setopt(curl, CURLOPT_CAPATH, "/Users/ygorelik/ydk-gen/scripts/community/openssl");
-//#endif
-//#ifdef SKIP_HOSTNAME_VERIFICATION
-        /*
-         * If the site you're connecting to uses a different host name that what
-         * they have mentioned in their server certificate's commonName (or
-         * subjectAltName) fields, libcurl will refuse to connect. You can skip
-         * this check, but this will make the connection less secure.
-         */
-        curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L);
-//#endif
-        curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L);
+        if (!(ssl_options & SSL_PEER_VERIFICATION))
+        {
+            /*
+             * If you want to connect to a site who isn't using a certificate that is
+             * signed by one of the certs in the CA bundle you have, you can skip the
+             * verification of the server's certificate. This makes the connection
+             * A LOT LESS SECURE.
+             *
+             * If you have a CA cert for the server stored someplace else than in the
+             * default bundle, then the CURLOPT_CAPATH option might come handy for
+             * you.
+             */
+            curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
+            YLOG_INFO("SSL peer verification is disabled!");
+        }
+        if (!(ssl_options & SSL_HOSTNAME_VERIFICATION))
+        {
+            /*
+             * If the site you're connecting to uses a different host name that what
+             * they have mentioned in their server certificate's commonName (or
+             * subjectAltName) fields, libcurl will refuse to connect. You can skip
+             * this check, but this will make the connection less secure.
+             */
+            curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L);
+            YLOG_INFO("SSL host name verification is disabled!");
+        }
     }
 
     header_options_list = curl_slist_append(header_options_list, ("Content-Type: " + encoding).c_str());

diff --git a/sdk/cpp/core/src/restconf_client.hpp b/sdk/cpp/core/src/restconf_client.hpp
@@ -29,11 +29,16 @@ struct curl_slist;
 
 namespace ydk
 {
+
+// Flags to control SSL verification
+#define SSL_PEER_VERIFICATION 0x01
+#define SSL_HOSTNAME_VERIFICATION 0x02
+
 class RestconfClient
 {
     public:
         RestconfClient(const std::string & address, const std::string & username, const std::string & password,
-                    int port, const std::string & encoding);
+                    int port, const std::string & encoding, uint ssl_options=0);
         ~RestconfClient();
 
         std::string execute(const std::string & yfilter, const std::string & url, const std::string & payload) const;
@@ -50,6 +55,7 @@ class RestconfClient
         std::string base_url;
         std::string encoding;
         std::string protocol;
+        uint ssl_options;
 };
 }
 

diff --git a/sdk/python/core/docsgen/developer_guide.rst b/sdk/python/core/docsgen/developer_guide.rst
@@ -14,4 +14,5 @@ Developer Guide
     guides/deviation.rst
     guides/path.rst
     guides/opendaylight.rst
+    guides/restconf_https.rst
     guides/backward_compatibility.rst
diff --git a/sdk/python/core/docsgen/guides/restconf_https.rst b/sdk/python/core/docsgen/guides/restconf_https.rst
@@ -0,0 +1,98 @@
+..
+  # ***************************************************************
+  # YDK - YANG Development Kit 
+  # Copyright 2020 Yan Gorelik, YDK Solutions. All rights reserved
+  # ***************************************************************
+  # Licensed to the Apache Software Foundation (ASF) under one
+  # or more contributor license agreements.  See the NOTICE file
+  # distributed with this work for additional information
+  # regarding copyright ownership.  The ASF licenses this file
+  # to you under the Apache License, Version 2.0 (the
+  # "License"); you may not use this file except in compliance
+  # with the License.  You may obtain a copy of the License at
+  #
+  #   http:#www.apache.org/licenses/LICENSE-2.0
+  #
+  # Unless required by applicable law or agreed to in writing,
+  # software distributed under the License is distributed on an
+  # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  # KIND, either express or implied.  See the License for the
+  # specific language governing permissions and limitations
+  # under the License.
+  # ***************************************************************
+
+Using Restconf with HTTPS
+=========================
+
+By default the RestconfServiceProvider initializes to support HTTP non-secure protocol.
+But YDK also provides partial support for HTTPS protocol. Here 'partial' means that YDK is capable communicate over secure protocol,
+provides data encryption, checks Restconf server CA certificate, but the peer and host name verifications are permanently disabled.
+This limitation should be addressed in future YDK releases.
+
+
+CA Certificate Installation
+---------------------------
+
+In order to enable HTTPS protocol, the user must obtain and install the Restconf server CA certificate on application server.
+On Ubuntu the installation procedure is as followed:
+
+.. code-block:: sh
+
+    cd /usr/local/share/ca-certificates/
+    sudo mkdir ydk
+    cp ~/myrestconf.crt ydk/
+    # Make sure the permissions are OK (755 for the folder, 644 for the file)
+    sudo update-ca-certificates
+    # In the output of the last command check that the certificate was added
+
+The installation procedure on CentOS-7:
+
+.. code-block:: sh
+
+    sudo cp ~/myrestconf.crt /etc/pki/ca-trust/source/anchors/
+    sudo update-ca-trust
+
+Getting Mac-OSX to trust self-signed SSL Certificates:
+
+ 1. Locate your CA certificate file.
+ 2. Open up Keychain Access.
+ 3. Drag your certificate into Keychain Access.
+ 4. Go into the Certificates section and locate the certificate you just added.
+ 5. Double click on it, enter the trust section and under “When using this certificate” select “Always Trust”.
+
+Code Snippet
+------------
+
+In the application the user must explicitly specify HTTPS protocol in the host address.
+The following example shows, how the RestconfServiceProvider is used to read names of all interfaces from secure Restconf server:
+
+.. code-block:: python
+ :linenos:
+
+ #!/usr/bin/env python
+ #
+ from ydk.services import CRUDService
+ from ydk.providers import RestconfServiceProvider, NetconfServiceProvider
+ from ydk.types import EncodingFormat
+ from ydk.path import Repository
+ from ydk.filters import YFilter
+ from ydk.models.openconfig import openconfig_interfaces
+
+ if __name__ == '__main__':
+
+    repo = Repository('/Users/ygorelik/.ydk/sbx-iosxr-mgmt.cisco.com')
+    provider = RestconfServiceProvider(
+        repo,
+        'https://ios-xe-mgmt.cisco.com',   # Add 'https://' prefix to the host name or address
+        'developer',
+        'C1sco12345',
+        9443,    # HTTPS port
+        EncodingFormat.JSON)
+
+    interfaces = openconfig_interfaces.Interfaces()
+
+    crud = CRUDService()
+    all_interfaces = crud.read(provider, interfaces)
+
+    for intf in all_interfaces.interface:
+        print(intf.name)