Allow SCIM without saml

docs/reference/spar-braindump.md

@@ -21,6 +21,25 @@ documentation answering your questions, look here!
 - if you want to work on our saml/scim implementation and do not have access to [https://github.com/zinfra/backend-issues/issues?q=is%3Aissue+is%3Aopen+label%3Aspar] and [https://github.com/wireapp/design-specs/tree/master/Single%20Sign%20On], please get in touch with us.
 
 
+## design considerations
+
+### SCIM without SAML.
+
+Before https://github.com/wireapp/wire-server/pull/1200, scim tokens could only be added to teams that already had exactly one SAML IdP.  Now, we also allow SAML-less teams to have SCIM provisioning.  This is an alternative to onboarding via team-settings and produces user accounts that are authenticated with email and password.  (Phone may or may not work, but is not officially supported.)
+
+The way this works is different from team-settings: we don't send invites, but we create active users immediately the moment the SCIM user post is processed.  The new thing is that the created user has neither email nor phone nor a SAML identity, nor a password.
+
+How does this work?
+
+**email:** If no SAML IdP is present, SCIM user posts must contain an externalId that is an email address.  This email address is not added to the newly created user, because it has not been validated.  Instead, the flow for changing an email address is triggered in brig: an email is sent to the address containing a validation key, and once the user completes the flow, brig will add the email address to the user.  We had to add very little code for this in this PR, it's all an old feature.
+
+When SCIM user gets are processed, in order to reconstruct the externalId from the user spar is retrieving from brig, we introduce a new json object for the `sso_id` field that looks like this: `{'scim_external_id': '[email protected]'}`.
+
+In order to find users that have email addresses pending validation, we introduce a new table in spar's cassandra called `scim_external_ids`, in analogy to `user`.  We have tried to use brig's internal `GET /i/user&email=...`, but that also finds pending email addresses, and there are corner cases when changing email addresses and waiting for the new address to be validated and the old to be removed...  that made this approach seem infeasible.
+
+**password:** once the user has validated their email address, they need to trigger the "forgot password" flow -- also old code.
+
+
 ## operations
 
 ### enabling / disabling the sso feature for a team

libs/brig-types/src/Brig/Types/Intra.hs

@@ -64,7 +64,7 @@ instance ToJSON AccountStatus where
   toJSON Deleted = String "deleted"
   toJSON Ephemeral = String "ephemeral"
 
-data AccountStatusResp = AccountStatusResp AccountStatus
+data AccountStatusResp = AccountStatusResp {fromAccountStatusResp :: AccountStatus}
 
 instance ToJSON AccountStatusResp where
   toJSON (AccountStatusResp s) = object ["status" .= s]

libs/wire-api/src/Wire/API/User.hs

@@ -694,10 +694,9 @@ parseNewUserOrigin pass uid ssoid o = do
     (Nothing, Nothing, Just a, Nothing, Nothing) -> return . Just . NewUserOriginTeamUser $ NewTeamCreator a
     (Nothing, Nothing, Nothing, Just _, Just t) -> return . Just . NewUserOriginTeamUser $ NewTeamMemberSSO t
     (Nothing, Nothing, Nothing, Nothing, Nothing) -> return Nothing
-    (_, _, _, _, _) ->
-      fail $
-        "team_code, team, invitation_code, sso_id are mutually exclusive\
-        \ and sso_id, team_id must be either both present or both absent."
+    (_, _, _, Just _, Nothing) -> fail "sso_id, team_id must be either both present or both absent."
+    (_, _, _, Nothing, Just _) -> fail "sso_id, team_id must be either both present or both absent."
+    _ -> fail "team_code, team, invitation_code, sso_id, and the pair (sso_id, team_id) are mutually exclusive"
   case (result, pass, uid) of
     (_, _, Just SSOIdentity {}) -> pure result
     (Just (NewUserOriginTeamUser _), Nothing, _) -> fail "all team users must set a password on creation"
@@ -729,7 +728,8 @@ data NewTeamUser
   = -- | requires email address
     NewTeamMember InvitationCode
   | NewTeamCreator BindingNewTeamUser
-  | NewTeamMemberSSO TeamId
+  | -- | sso: users with saml credentials and/or created via scim
+    NewTeamMemberSSO TeamId
   deriving stock (Eq, Show, Generic)
   deriving (Arbitrary) via (GenericUniform NewTeamUser)
 

libs/wire-api/src/Wire/API/User/Identity.hs

@@ -247,24 +247,34 @@ isValidPhone = either (const False) (const True) . parseOnly e164
 -- Morally this is the same thing as 'SAML.UserRef', but we forget the
 -- structure -- i.e. we just store XML-encoded SAML blobs. If the structure
 -- of those blobs changes, Brig won't have to deal with it, only Spar will.
-data UserSSOId = UserSSOId
-  { -- | An XML blob pointing to the identity provider that can confirm
-    -- user's identity.
-    userSSOIdTenant :: Text,
-    -- | An XML blob specifying the user's ID on the identity provider's side.
-    userSSOIdSubject :: Text
-  }
+--
+-- FUTUREWORK: rename the data type to @UserSparId@ (not the two constructors, those are ok).
+data UserSSOId
+  = UserSSOId
+      -- An XML blob pointing to the identity provider that can confirm
+      -- user's identity.
+      Text
+      -- An XML blob specifying the user's ID on the identity provider's side.
+      Text
+  | UserScimExternalId
+      Text
   deriving stock (Eq, Show, Generic)
   deriving (Arbitrary) via (GenericUniform UserSSOId)
 
 instance ToJSON UserSSOId where
-  toJSON (UserSSOId tenant subject) = object ["tenant" .= tenant, "subject" .= subject]
+  toJSON = \case
+    UserSSOId tenant subject -> object ["tenant" .= tenant, "subject" .= subject]
+    UserScimExternalId eid -> object ["scim_external_id" .= eid]
 
 instance FromJSON UserSSOId where
-  parseJSON = withObject "UserSSOId" $ \obj ->
-    UserSSOId
-      <$> obj .: "tenant"
-      <*> obj .: "subject"
+  parseJSON = withObject "UserSSOId" $ \obj -> do
+    mtenant <- obj .:? "tenant"
+    msubject <- obj .:? "subject"
+    meid <- obj .:? "scim_external_id"
+    case (mtenant, msubject, meid) of
+      (Just tenant, Just subject, Nothing) -> pure $ UserSSOId tenant subject
+      (Nothing, Nothing, Just eid) -> pure $ UserScimExternalId eid
+      _ -> fail "either need tenant and subject, or scim_external_id, but not both"
 
 -- | If the budget for SMS and voice calls for a phone number
 -- has been exhausted within a certain time frame, this timeout

services/brig/src/Brig/API/Internal.hs

@@ -29,6 +29,7 @@ import Brig.API.Handler
 import qualified Brig.API.IdMapping as IdMapping
 import Brig.API.Types
 import qualified Brig.API.User as API
+import Brig.API.Util (validateHandle)
 import Brig.App
 import qualified Brig.Data.User as Data
 import Brig.Options hiding (internalEvents, sesQueue)
@@ -58,6 +59,7 @@ import Network.Wai.Routing
 import Network.Wai.Utilities as Utilities
 import Network.Wai.Utilities.Response (json)
 import Network.Wai.Utilities.ZAuth (zauthConnId, zauthUserId)
+import Wire.API.User
 import Wire.API.User.RichInfo
 
 ---------------------------------------------------------------------------
@@ -173,6 +175,10 @@ sitemap = do
       .&. accept "application" "json"
       .&. jsonRequest @UserSSOId
 
+  delete "/i/users/:uid/sso-id" (continue deleteSSOIdH) $
+    capture "uid"
+      .&. accept "application" "json"
+
   put "/i/users/:uid/managed-by" (continue updateManagedByH) $
     capture "uid"
       .&. accept "application" "json"
@@ -183,6 +189,22 @@ sitemap = do
       .&. accept "application" "json"
       .&. jsonRequest @RichInfoUpdate
 
+  put "/i/users/:uid/handle" (continue updateHandleH) $
+    capture "uid"
+      .&. accept "application" "json"
+      .&. jsonRequest @HandleUpdate
+
+  put "/i/users/:uid/name" (continue updateUserNameH) $
+    capture "uid"
+      .&. accept "application" "json"
+      .&. jsonRequest @NameUpdate
+
+  get "/i/users/:uid/rich-info" (continue getRichInfoH) $
+    capture "uid"
+
+  head "/i/users/handles/:handle" (continue checkHandleInternalH) $
+    capture "handle"
+
   post "/i/clients" (continue internalListClientsH) $
     accept "application" "json"
       .&. jsonRequest @UserSet
@@ -433,7 +455,14 @@ addPhonePrefixH (_ ::: req) = do
 updateSSOIdH :: UserId ::: JSON ::: JsonRequest UserSSOId -> Handler Response
 updateSSOIdH (uid ::: _ ::: req) = do
   ssoid :: UserSSOId <- parseJsonBody req
-  success <- lift $ Data.updateSSOId uid ssoid
+  success <- lift $ Data.updateSSOId uid (Just ssoid)
+  if success
+    then return empty
+    else return . setStatus status404 $ plain "User does not exist or has no team."
+
+deleteSSOIdH :: UserId ::: JSON -> Handler Response
+deleteSSOIdH (uid ::: _) = do
+  success <- lift $ Data.updateSSOId uid Nothing
   if success
     then return empty
     else return . setStatus status404 $ plain "User does not exist or has no team."
@@ -457,6 +486,44 @@ updateRichInfo uid rup = do
   -- Intra.onUserEvent uid (Just conn) (richInfoUpdate uid ri)
   lift $ Data.updateRichInfo uid (RichInfoAssocList richInfo)
 
+getRichInfoH :: UserId -> Handler Response
+getRichInfoH uid = json <$> getRichInfo uid
+
+getRichInfo :: UserId -> Handler RichInfo
+getRichInfo uid = RichInfo . fromMaybe emptyRichInfoAssocList <$> lift (API.lookupRichInfo uid)
+
+updateHandleH :: UserId ::: JSON ::: JsonRequest HandleUpdate -> Handler Response
+updateHandleH (uid ::: _ ::: body) = empty <$ (updateHandle uid =<< parseJsonBody body)
+
+updateHandle :: UserId -> HandleUpdate -> Handler ()
+updateHandle uid (HandleUpdate handleUpd) = do
+  handle <- validateHandle handleUpd
+  API.changeHandle uid Nothing handle !>> changeHandleError
+
+updateUserNameH :: UserId ::: JSON ::: JsonRequest NameUpdate -> Handler Response
+updateUserNameH (uid ::: _ ::: body) = empty <$ (updateUserName uid =<< parseJsonBody body)
+
+updateUserName :: UserId -> NameUpdate -> Handler ()
+updateUserName uid (NameUpdate nameUpd) = do
+  name <- either (const $ throwStd invalidUser) pure $ mkName nameUpd
+  let uu =
+        UserUpdate
+          { uupName = Just name,
+            uupPict = Nothing,
+            uupAssets = Nothing,
+            uupAccentId = Nothing
+          }
+  lift (Data.lookupUser uid) >>= \case
+    Just _ -> lift $ API.updateUser uid Nothing uu
+    Nothing -> throwStd invalidUser
+
+checkHandleInternalH :: Text -> Handler Response
+checkHandleInternalH =
+  API.checkHandle >=> \case
+    API.CheckHandleInvalid -> throwE (StdError invalidHandle)
+    API.CheckHandleFound -> pure $ setStatus status200 empty
+    API.CheckHandleNotFound -> pure $ setStatus status404 empty
+
 getContactListH :: JSON ::: UserId -> Handler Response
 getContactListH (_ ::: uid) = do
   contacts <- lift $ API.lookupContactList uid

services/brig/src/Brig/API/Public.hs

@@ -1096,7 +1096,7 @@ instance ToJSON GetActivationCodeResp where
 updateUserH :: UserId ::: ConnId ::: JsonRequest Public.UserUpdate -> Handler Response
 updateUserH (uid ::: conn ::: req) = do
   uu <- parseJsonBody req
-  lift $ API.updateUser uid conn uu
+  lift $ API.updateUser uid (Just conn) uu
   return empty
 
 changePhoneH :: UserId ::: ConnId ::: JsonRequest Public.PhoneUpdate -> Handler Response
@@ -1177,7 +1177,7 @@ changeHandleH (u ::: conn ::: req) = do
 changeHandle :: UserId -> ConnId -> Public.HandleUpdate -> Handler ()
 changeHandle u conn (Public.HandleUpdate h) = do
   handle <- API.validateHandle h
-  API.changeHandle u conn handle !>> changeHandleError
+  API.changeHandle u (Just conn) handle !>> changeHandleError
 
 beginPasswordResetH :: JSON ::: JsonRequest Public.NewPasswordReset -> Handler Response
 beginPasswordResetH (_ ::: req) = do

services/brig/src/Brig/API/User.hs

@@ -335,10 +335,10 @@ checkRestrictedUserCreation new = do
 -- FUTUREWORK: this and other functions should refuse to modify a ManagedByScim user. See
 -- {#SparBrainDump}  https://github.com/zinfra/backend-issues/issues/1632
 
-updateUser :: UserId -> ConnId -> UserUpdate -> AppIO ()
-updateUser uid conn uu = do
+updateUser :: UserId -> Maybe ConnId -> UserUpdate -> AppIO ()
+updateUser uid mconn uu = do
   Data.updateUser uid uu
-  Intra.onUserEvent uid (Just conn) (profileUpdated uid uu)
+  Intra.onUserEvent uid mconn (profileUpdated uid uu)
 
 -------------------------------------------------------------------------------
 -- Update Locale
@@ -359,8 +359,8 @@ changeManagedBy uid conn (ManagedByUpdate mb) = do
 --------------------------------------------------------------------------------
 -- Change Handle
 
-changeHandle :: UserId -> ConnId -> Handle -> ExceptT ChangeHandleError AppIO ()
-changeHandle uid conn hdl = do
+changeHandle :: UserId -> Maybe ConnId -> Handle -> ExceptT ChangeHandleError AppIO ()
+changeHandle uid mconn hdl = do
   when (isBlacklistedHandle hdl) $
     throwE ChangeHandleInvalid
   usr <- lift $ Data.lookupUser uid
@@ -374,7 +374,7 @@ changeHandle uid conn hdl = do
       claimed <- lift $ claimHandle (userId u) (userHandle u) hdl
       unless claimed $
         throwE ChangeHandleExists
-      lift $ Intra.onUserEvent uid (Just conn) (handleUpdated uid hdl)
+      lift $ Intra.onUserEvent uid mconn (handleUpdated uid hdl)
 
 --------------------------------------------------------------------------------
 -- Check Handle

services/brig/src/Brig/Data/User.hs

@@ -238,7 +238,7 @@ updateEmail u e = retry x5 $ write userEmailUpdate (params Quorum (e, u))
 updatePhone :: UserId -> Phone -> AppIO ()
 updatePhone u p = retry x5 $ write userPhoneUpdate (params Quorum (p, u))
 
-updateSSOId :: UserId -> UserSSOId -> AppIO Bool
+updateSSOId :: UserId -> Maybe UserSSOId -> AppIO Bool
 updateSSOId u ssoid = do
   mteamid <- lookupUserTeam u
   case mteamid of
@@ -549,7 +549,7 @@ userEmailUpdate = "UPDATE user SET email = ? WHERE id = ?"
 userPhoneUpdate :: PrepQuery W (Phone, UserId) ()
 userPhoneUpdate = "UPDATE user SET phone = ? WHERE id = ?"
 
-userSSOIdUpdate :: PrepQuery W (UserSSOId, UserId) ()
+userSSOIdUpdate :: PrepQuery W (Maybe UserSSOId, UserId) ()
 userSSOIdUpdate = "UPDATE user SET sso_id = ? WHERE id = ?"
 
 userManagedByUpdate :: PrepQuery W (ManagedBy, UserId) ()

services/brig/src/Brig/Team/API.hs

@@ -283,7 +283,7 @@ deleteInvitationH (_ ::: uid ::: tid ::: iid) = do
 deleteInvitation :: UserId -> TeamId -> InvitationId -> Handler ()
 deleteInvitation uid tid iid = do
   ensurePermissions uid tid [Team.AddTeamMember]
-  lift $ DB.deleteInvitation tid iid
+  DB.deleteInvitation tid iid
 
 listInvitationsH :: JSON ::: UserId ::: TeamId ::: Maybe InvitationId ::: Range 1 500 Int32 -> Handler Response
 listInvitationsH (_ ::: uid ::: tid ::: start ::: size) = do

services/spar/package.yaml

@@ -143,6 +143,7 @@ executables:
       - silently
       - spar
       - stm
+      - tasty-hunit
       - tinylog
       - wai
       - wai-extra

services/spar/schema/src/Main.hs

@@ -24,6 +24,7 @@ import qualified System.Logger.Extended as Log
 import Util.Options
 import qualified V0
 import qualified V1
+import qualified V10
 import qualified V2
 import qualified V3
 import qualified V4
@@ -51,7 +52,8 @@ main = do
       V6.migration,
       V7.migration,
       V8.migration,
-      V9.migration
+      V9.migration,
+      V10.migration
       -- When adding migrations here, don't forget to update
       -- 'schemaVersion' in Spar.Data
 

services/spar/schema/src/V10.hs

@@ -0,0 +1,37 @@
+-- This file is part of the Wire Server implementation.
+--
+-- Copyright (C) 2020 Wire Swiss GmbH <[email protected]>
+--
+-- This program is free software: you can redistribute it and/or modify it under
+-- the terms of the GNU Affero General Public License as published by the Free
+-- Software Foundation, either version 3 of the License, or (at your option) any
+-- later version.
+--
+-- This program is distributed in the hope that it will be useful, but WITHOUT
+-- ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+-- FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+-- details.
+--
+-- You should have received a copy of the GNU Affero General Public License along
+-- with this program. If not, see <https://www.gnu.org/licenses/>.
+
+module V10
+  ( migration,
+  )
+where
+
+import Cassandra.Schema
+import Imports
+import Text.RawString.QQ
+
+migration :: Migration
+migration = Migration 10 "Add table for mapping scim external ids to brig user ids" $ do
+  void $
+    schema'
+      [r|
+        CREATE TABLE if not exists scim_external_ids
+          ( external  text
+          , user      uuid
+          , primary key (external)
+          ) with compaction = {'class': 'LeveledCompactionStrategy'};
+      |]

services/spar/spar.cabal

@@ -4,7 +4,7 @@ cabal-version: 1.12
 --
 -- see: https://github.com/sol/hpack
 --
--- hash: c4215bccf7e235dad19c5eb68954faf664f92ff0b3d7aeacdfb6ac21b448503b
+-- hash: 88a250ccd05fb0ad0b42a8b133068a52e559f6b5e00efb0df3098445c92f2a45
 
 name:           spar
 version:        0.1
@@ -207,6 +207,7 @@ executable spar-integration
       Test.Spar.Scim.UserSpec
       Util
       Util.Core
+      Util.Email
       Util.Scim
       Util.Types
       Paths_spar
@@ -277,6 +278,7 @@ executable spar-integration
     , stm
     , string-conversions
     , swagger2
+    , tasty-hunit
     , text
     , text-latin1
     , time
@@ -306,6 +308,7 @@ executable spar-schema
   other-modules:
       V0
       V1
+      V10
       V2
       V3
       V4