Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PLANNING ISSUE : Reply via Twitter . #2671

Closed
sagarpreet-chadha opened this issue Apr 28, 2018 · 72 comments · Fixed by #3175
Closed

PLANNING ISSUE : Reply via Twitter . #2671

sagarpreet-chadha opened this issue Apr 28, 2018 · 72 comments · Fixed by #3175
Assignees
@namangupta01
Labels
feature explains that the issue is to add a new feature in progress more-detail-please issue lacks proper description and perhaps needs code links or the location of the problem planning Planning issues! summer-of-code

Comments

@sagarpreet-chadha
Copy link
Contributor

sagarpreet-chadha commented Apr 28, 2018 •
edited
Loading

Please describe the problem (or idea)

This is a planning issue for reply by twitter : feature .

Let's first discuss the best approach and then we shall break this into smaller independent issues .

More Details 🔜 !

=> Email Integration #2672 (@namangupta01 )
@sagarpreet-chadha sagarpreet-chadha added help wanted requires help by anyone willing to contribute summer-of-code feature explains that the issue is to add a new feature more-detail-please issue lacks proper description and perhaps needs code links or the location of the problem and removed help wanted requires help by anyone willing to contribute labels Apr 28, 2018
@jywarren
Copy link
Member

jywarren commented Apr 29, 2018 via email

@jywarren jywarren added the planning Planning issues! label May 3, 2018
@sagarpreet-chadha
Copy link
Contributor Author

@namangupta01 , @jywarren ...lets start the discussion on this ? When are you both free ?

@namangupta01
Copy link
Member

namangupta01 commented Jun 14, 2018 via email

@namangupta01
Copy link
Member

I have some implementation in my mind...will post in a while.

@jywarren
Copy link
Member

jywarren commented Jun 14, 2018 via email

@namangupta01
Copy link
Member

Spamming is the major issue. I was reading documentation from past few days. I was also wondering if we implement as you said "response tweets have to be liked" i found a major issue.
The issue is that whenever a user retweets on tweet (which is a node) we can know about the retweet through apis but when the retweet is liked we can't know about it. The only way to know about it is to search for every retweet status after certain interval of time which is very bad because lets suppose we have 10k retweets then we have to search for 10k retweets status for being liked in certain interval of time. And also this 10k figure will continue to rise over time.

@namangupta01
Copy link
Member

And also one main point which could leads to no spamming is that we can only add the tweets of those users whose twitter handle is associated to any publiclab account just like we were doing in reply-by-email where email of email sender is first checked for associativity with the publiclab account.
This could leads to no spamming.

@sagarpreet-chadha
Copy link
Contributor Author

@namangupta01 ...great points !

  • Spamming is the issue here , i guess the only moderation technique i can also think of is that a moderator has to approve the comment on a tweet by liking it (Similar to how things work in publiclab.org ) .
    => moderator can be people with PublicLab's twitter account OR a PL community person who has liked from his own account .

  • I guess the point is to bring a member who is new to the community , so bringing comments of only existing people will solve half the problem .

What do you think ?

@namangupta01
Copy link
Member

Great thinking! But the only point i am concern with is that we have to check explicitly for the tweet's being liked in every request for all the tweets on the publiclab tweets which can become very large request as the tweets increases.

@namangupta01
Copy link
Member

And most of the time we will be making request for the tweets for nothing...that was my concern. What do you think?

@namangupta01
Copy link
Member

namangupta01 commented Jun 20, 2018 •
edited
Loading

However moderation technique on the publiclab website interface would work great.
Like approving comments ?

@namangupta01
Copy link
Member

@jywarren As email feature is almost complete we could discuss over it?

@jywarren
Copy link
Member

jywarren commented Jun 26, 2018 via email

@namangupta01
Copy link
Member

@jywarren Can we put node_id in tweets? So that it can be easily identified?

@namangupta01
Copy link
Member

namangupta01 commented Jul 15, 2018 •
edited
Loading

@jywarren Is it possible to put node_id in tweets?

@jywarren
Copy link
Member

jywarren commented Jul 15, 2018 via email

@namangupta01
Copy link
Member

@jywarren So publiclab.org/n/____ will present directly in tweet or there are some other links which redirect to this link as before?
ex:
screen shot 2018-07-17 at 2 07 32 am

@namangupta01
Copy link
Member

I was researching and found that all the links that comes in tweet through twitter api is a twitter link which gets redirected to actual page. I was wondering if we use link to find the node and tweet contains more than one public.org/n/__(one for referencing post on public lab and other may be used in any of node content on public lab post itself) then in that case comment may get added to the wrong node because we donot know what the actual node we should add this comment? What do you think?
screen shot 2018-07-17 at 3 40 13 am
screen shot 2018-07-17 at 3 40 23 am

@namangupta01
Copy link
Member

@jywarren So, by which way should i implement it ? using node_id or the other way. I am finalising the way of implementation.

@jywarren
Copy link
Member

jywarren commented Jul 17, 2018 via email

@namangupta01
Copy link
Member

I am going to use twitter gem using which we can also do auto tweet --just let you know that we can do autotweeting without using any service. I will look for the way to format.

@jywarren
Copy link
Member

OK, so what we're thinking of is:

  1. start by implementing only replies by "recognized" twitter users, using the OAuth login system
  2. attempt to find tweets by isolating publiclab.org/n/_____
  3. could we display all comments in a list only to moderators and allow them to be "approved" somehow? Can you think through how that might work?
  4. if we do start allowing all tweeted replies, let's consider having a way to "turn it off" if we start to get a LOT of spam, that way we can try things out.
  5. are there ways to display an embedded tweet with its replies, and then, if there are tweeted replies, notify the original poster, but not actually copy them in?

@namangupta01
Copy link
Member

namangupta01 commented Jul 19, 2018 •
edited
Loading

I just had a call with @SidharthBansal. I asked about this. So we are not saving twitter handle right now which is require to recognise the users. I think we should save it somewhere so we can recognise it. @jywarren ?

@SidharthBansal
Copy link
Member

@namangupta01 you can migrate the user tag model and add handler field.
If you need any further help feel free to contact me.
Thanks

@namangupta01
Copy link
Member

Pseudocode:

  1. Fetch latest replies from Twitter :
  2. For every reply in replies :
    check if it belongs to any node or post using publiclab.org/n/_____ and if that node exists then find if the user exist using tweet replier. If user exists add the comment.
tweets = client.search("to:publiclab", since_id: 23423423).collect do |tweet|
 node = Node.where(some_condition) #To find if node exists
 if node.any?
   node = node.first
   tweet_user = tweet.user
   UserTag.all.each do |user_tag|
     if(some_condtion_to_check if user handle exists)
       add_comment
     end
   end
end
end

@namangupta01
Copy link
Member

namangupta01 commented Jul 20, 2018 via email

@namangupta01 namangupta01 mentioned this issue Jul 21, 2018
Closed
@namangupta01
Copy link
Member

HI @jywarren I have added migration to save JSON for omniauth response in #3117 just to give it a start...let me know if needs any change?

@SidharthBansal SidharthBansal mentioned this issue Jul 21, 2018
Open
@namangupta01
Copy link
Member

@jywarren any review and feedback regarding pseudocode and pr?

@namangupta01
Copy link
Member

i am hoping to complete this as soon as possible.

@jywarren
Copy link
Member

Can you share a typical JSON output that'd be stored in this new column, so we can see? Thanks!

For the pseudocode, could you do something like:

since = comments.where(since_id not null).last # get most recent comment-from-tweet time
tweets = client.search("to:publiclab", since_id: 23423423).collect do |tweet|
 node = Node.where(some_condition) #To find if node exists
 if node.any?
   node = node.first
   tweet_user = tweet.user
   ruser = UserTag.find_by(twitter_id: tweet.user)
   add_comment(since_id: since, ruser) unless ruser.nil? 
end
end

It might be more optimal?

@namangupta01
Copy link
Member

Here's is the sample JSON:

{
   "provider":"twitter",
   "uid":"849503183529545728",
   "info":{
      "nickname":"itsmenamangupta",
      "name":"Naman Gupta",
      "email":"[email protected]",
      "location":"",
      "image":"http://pbs.twimg.com/profile_images/950001110378401793/mBAtBXyI_normal.jpg",
      "description":"GSoC'18 @PublicLab",
      "urls":{
         "Website":"https://t.co/fqLSxIlYFH",
         "Twitter":"https://twitter.com/itsmenamangupta"
      }
   },
   "credentials":{
      "token":"849503183529545728-fgqcxtgihvrrKd2icj67QpspT5bKntD",
      "secret":"ZDTD5HsexliY2Tnrv6aSEVSqPVvbDLY2YDW9S5VeDmmyK"
   },
   "extra":{
      "access_token":{
         "token":"849503183529545728-fgqcxtgihvrrKd2icj67QpspT5bKntD",
         "secret":"ZDTD5HsexliY2Tnrv6aSEVSqPVvbDLY2YDW9S5VeDmmyK",
         "consumer":{
            "key":"H59J9kJBVQvtZvWhr2SUAht4o",
            "secret":"GEQOygPITjGj2Imzg5ALJeCqE0GdQU2GAmTYOS9kEPcLVlMxQu",
            "options":{
               "signature_method":"HMAC-SHA1",
               "request_token_path":"/oauth/request_token",
               "authorize_path":"/oauth/authenticate",
               "access_token_path":"/oauth/access_token",
               "proxy":null,
               "scheme":"header",
               "http_method":"post",
               "debug_output":null,
               "oauth_version":"1.0",
               "site":"https://api.twitter.com"
            },
            "debug_output":null,
            "http":{
               "address":"api.twitter.com",
               "port":443,
               "local_host":null,
               "local_port":null,
               "curr_http_version":"1.1",
               "keep_alive_timeout":2,
               "last_communicated":null,
               "close_on_empty_response":false,
               "socket":null,
               "started":false,
               "open_timeout":30,
               "read_timeout":30,
               "continue_timeout":null,
               "debug_output":null,
               "proxy_from_env":true,
               "proxy_uri":null,
               "proxy_address":null,
               "proxy_port":null,
               "proxy_user":null,
               "proxy_pass":null,
               "use_ssl":true,
               "ssl_context":{
                  "verify_mode":0,
                  "verify_hostname":true
               },
               "ssl_session":{

               },
               "sspi_enabled":false,
               "ca_file":null,
               "ca_path":null,
               "cert":null,
               "cert_store":null,
               "ciphers":null,
               "key":null,
               "ssl_timeout":null,
               "ssl_version":null,
               "verify_callback":null,
               "verify_depth":null,
               "verify_mode":0
            },
            "http_method":"post",
            "uri":"https://api.twitter.com"
         },
         "params":{
            "oauth_token":"849503183529545728-fgqcxtgihvrrKd2icj67QpspT5bKntD",
            "oauth_token_secret":"ZDTD5HsexliY2Tnrv6aSEVSqPVvbDLY2YDW9S5VeDmmyK",
            "user_id":"849503183529545728",
            "screen_name":"itsmenamangupta"
         },
         "response":{
            "cache-control":[
               "no-cache, no-store, must-revalidate, pre-check=0, post-check=0"
            ],
            "connection":[
               "close"
            ],
            "content-disposition":[
               "attachment; filename=json.json"
            ],
            "content-length":[
               "664"
            ],
            "content-type":[
               "application/json;charset=utf-8"
            ],
            "date":[
               "Sat, 21 Jul 2018 10:51:03 GMT"
            ],
            "expires":[
               "Tue, 31 Mar 1981 05:00:00 GMT"
            ],
            "last-modified":[
               "Sat, 21 Jul 2018 10:51:03 GMT"
            ],
            "pragma":[
               "no-cache"
            ],
            "server":[
               "tsa_k"
            ],
            "set-cookie":[
               "personalization_id=\"v1_pEJH6fv6zGSET2PG1MYRCw==\"; Expires=Mon, 20 Jul 2020 10:51:03 GMT; Path=/; Domain=.twitter.com",
               "lang=en; Path=/",
               "guest_id=v1%3A153217026311648297; Expires=Mon, 20 Jul 2020 10:51:03 GMT; Path=/; Domain=.twitter.com"
            ],
            "status":[
               "200 OK"
            ],
            "strict-transport-security":[
               "max-age=631138519"
            ],
            "x-access-level":[
               "read-write-directmessages"
            ],
            "x-connection-hash":[
               "cf7caac1e8db2d46c6043fb2287ca32b"
            ],
            "x-content-type-options":[
               "nosniff"
            ],
            "x-frame-options":[
               "SAMEORIGIN"
            ],
            "x-rate-limit-limit":[
               "75"
            ],
            "x-rate-limit-remaining":[
               "71"
            ],
            "x-rate-limit-reset":[
               "1532170356"
            ],
            "x-response-time":[
               "238"
            ],
            "x-transaction":[
               "0085940a005f5411"
            ],
            "x-twitter-response-tags":[
               "BouncerExempt",
               "BouncerCompliant"
            ],
            "x-xss-protection":[
               "1; mode=block; report=https://twitter.com/i/xss_report"
            ]
         }
      },
      "raw_info":{
         "id":849503183529545728,
         "id_str":"849503183529545728",
         "name":"Naman Gupta",
         "screen_name":"itsmenamangupta",
         "location":"",
         "description":"GSoC'18 @PublicLab",
         "url":"https://t.co/fqLSxIlYFH",
         "entities":{
            "url":{
               "urls":[
                  {
                     "url":"https://t.co/fqLSxIlYFH",
                     "expanded_url":"http://www.namangupta.me",
                     "display_url":"namangupta.me",
                     "indices":[
                        0,
                        23
                     ]
                  }
               ]
            },
            "description":{
               "urls":[

               ]
            }
         },
         "protected":false,
         "followers_count":16,
         "friends_count":32,
         "listed_count":0,
         "created_at":"Wed Apr 05 06:05:16 +0000 2017",
         "favourites_count":86,
         "utc_offset":null,
         "time_zone":null,
         "geo_enabled":false,
         "verified":false,
         "statuses_count":42,
         "lang":"en",
         "contributors_enabled":false,
         "is_translator":false,
         "is_translation_enabled":false,
         "profile_background_color":"000000",
         "profile_background_image_url":"http://abs.twimg.com/images/themes/theme1/bg.png",
         "profile_background_image_url_https":"https://abs.twimg.com/images/themes/theme1/bg.png",
         "profile_background_tile":false,
         "profile_image_url":"http://pbs.twimg.com/profile_images/950001110378401793/mBAtBXyI_normal.jpg",
         "profile_image_url_https":"https://pbs.twimg.com/profile_images/950001110378401793/mBAtBXyI_normal.jpg",
         "profile_link_color":"FFFFFF",
         "profile_sidebar_border_color":"000000",
         "profile_sidebar_fill_color":"000000",
         "profile_text_color":"000000",
         "profile_use_background_image":false,
         "has_extended_profile":true,
         "default_profile":false,
         "default_profile_image":false,
         "following":false,
         "follow_request_sent":false,
         "notifications":false,
         "translator_type":"none",
         "suspended":false,
         "needs_phone_verification":false,
         "email":"[email protected]"
      }
   }
}

@namangupta01
Copy link
Member

Yes we can use the code specified by you, we will need twitter_id column in user_tag table?

@jywarren
Copy link
Member

jywarren commented Jul 23, 2018 via email

@namangupta01
Copy link
Member

Yes, it would make it expensive.
Adding direct username to column for all oauth services would help?

@grvsachdeva
Copy link
Member

Hi guys, just want to remind that we store, usernames for various social accounts in usertags too, so if you guys decide on switching to tables for storing username then, please open a follow-up issue to merge that system. Thanks!

@jywarren
Copy link
Member

jywarren commented Jul 23, 2018 via email

@SidharthBansal
Copy link
Member

SidharthBansal commented Jul 24, 2018 •
edited
Loading

Hi guys, just want to remind that we store, usernames for various social accounts in usertags too, so if you guys decide on switching to tables for storing username then, please open a follow-up issue to merge that system. Thanks!

Hi @Gauravano we are nowhere storing the usernames of the social media users till now. Currently only user id and provider name is saved.
@Gauravano can you please elaborate what issue are you pointing to? We are not making new tables. We are migrating the usertag.

Yes we can use the code specified by you, we will need twitter_id column in user_tag table?

yeah, the problem is you can't search by the json, probably... you could full text search but it's a bit expensive, right? @SidharthBansal - what do you think is a good approach here?

@jywarren, @namangupta01 we can store the username.

We do need to be careful that nobody falsely associates their account using the [twitter:jywarren] style tags, so maybe we need to combine these?

I have sent a pr in April for this case. No one is able to create any tag with name oauth:________
So, no need to worry about this issue. You can check this thing by going to the profile page.
Screenshot for better understanding.
image

Would it be worth auto-generating the [twitter:jywarren] style tag from the OA system?

@jywarren We are currenlty generating the OAuth usertags from OAuth ONLY. There is no other way to generate them.

Adding direct username to column for all oauth services would help?

Adding multiple columns again and again would not help. I will strongly suggest to store the entire hash and use what we need whensoever need arises. For example, after naman uses handler from auth hash. We can use the image from the image url to display on public lab profile page. We currently use Gravatar.

Related issue #3125
We are storing the handler. Some providers allow the user to change their handler. So, we need a method to change the auth hash either every time when the user logs in OR when the auth expires. The auth gives the expire date and time. So we can use that and refresh the auth hash accordingly. Please discuss this subissue stuff in #3125 so that we could stick to user_tag migration in this issue.

Thanks

@grvsachdeva
Copy link
Member

Hi @jywarren @SidharthBansal @namangupta01, I was referring to this system -

We provide PL users to enter their various social media usernames like this-
this_one

User add the social platform as a key and his username as a value-
user_2

After that usertag is added to his profile-
profile_tags

Thanks!

@namangupta01
Copy link
Member

@Gauravano I guess Using this will not be a good because i can be able to add any username i want ?

@namangupta01
Copy link
Member

As the @jywarren we should merge the OA system with it?

@jywarren
Copy link
Member

OK, storing the hash makes sense - why do the extra converting, right? Cool.

And +1 merging these systems. The earlier system is great but would benefit from integration here for the reasons above! Thanks, everyone!

@namangupta01
Copy link
Member

namangupta01 commented Jul 24, 2018 via email

@namangupta01
Copy link
Member

@jywarren @Gauravano @SidharthBansal can you guys please tell me how to get hash from the string saved in db because i am not able to find a way? Thanks

@SidharthBansal
Copy link
Member

We can follow approaches like https://stackoverflow.com/questions/35665266/rails-how-to-save-a-hash-to-the-db-and-using-it-as-a-hash-once-you-pull-it-for?rq=1
OR https://stackoverflow.com/questions/6694432/using-rails-serialize-to-save-hash-to-database
and go for similar links if they does not help
Thanks

@SidharthBansal
Copy link
Member

SidharthBansal commented Jul 24, 2018 •
edited
Loading

Hi @Gauravano I will highly recommend you to open another issue for merge of Social Media icons with OA and link it here so that the discussion will not be divergent.
Thanks for the nice suggestion.

@namangupta01
Copy link
Member

Hi @SidharthBansa i already tried the above links but they didnt work

@namangupta01
Copy link
Member

I got the problem..we have to explicitly convert the auth variable to hash to work it out.

@namangupta01 namangupta01 mentioned this issue Aug 1, 2018
Merged
5 tasks
@ghost ghost assigned namangupta01 Aug 1, 2018
@ghost ghost added the in progress label Aug 1, 2018
@sagarpreet-chadha sagarpreet-chadha mentioned this issue May 10, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@namangupta01 namangupta01

Labels
feature explains that the issue is to add a new feature in progress more-detail-please issue lacks proper description and perhaps needs code links or the location of the problem planning Planning issues! summer-of-code
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Added reply by tweet feature namangupta01/plots2
5 participants
@jywarren @sagarpreet-chadha @namangupta01 @grvsachdeva @SidharthBansal