Twitter Gaming: Tweepi.com Functionality Rip

There are plenty of twitter management tools out that will help automate / manage your twitter account for a few bucks a month.

While most products don’t add too much value I’ve found some particular features on Tweepi.com that have been pretty helpful. In particular there’s a stat called Follower Ratio which lets you know how likely a user is to follow you back; > 80% is a user thats probably worth following (and unfollowing later once we have a healthy count..will lose some followers from bot automation but majority of real users won’t unfollow back)

When using the free version of the site there is a tool that allows us to search up to 25 users using a comma seperated list. This definitely looks like a tasty target and upon checking the network tab we can see a clean POST with easily understood params and response. A test response is shown below

We can see from the response below that calculating Follower Ratio is as easy as followers_count/friends_count is all we need to process a user object in our script.

POST https://tweepi.com/data/get_users_pp/follow_by_copy_paste.json
....

RESPONSE:
{
   "total":1,
   "page_size":20,
   "users":[
      {
         "user_id":"*****",
         "screen_name":"*************",
         "location":"",
         "full_name":"*************",
         "last_tweet_time":"1969-12-31 16:00:00 -0800", // cute
         "followers_count":1,
         "friends_count":89,
         "statuses_count":0,
         "profile_img_url":"\/\/abs.twimg.com\/sticky\/default_profile_images\/default_profile_normal.png",
         "is_verified":false,
         "utc_offset":null,
         "bio":"it's hi astonishing ocean blue eyes that iadore the most but it's his contagious bright smile that transforms my orrowful frown into something joyous",
         "url":"",
         "lang":"en",
         "is_protected":false,
         "member_since":"2017-09-06 05:02:47 -0700",
         "listed_count":0,
         "favourites_count":0,
         "default_profile":true,
         "default_profile_image":true,
         "geo_enabled":false,
         "ui_last_updated":"2018-01-01 12:36:23 -0800",
         "is_follower":false,
         "is_friend":false,
         "follower_or_following":" ",
         "history_date":"2000-01-01 00:00:00 +0000",
         "is_safelisted":false,
         "tag_names":[

         ],
         "tag_dates":[

         ]
      }
   ]
}

As long as we can replay this request then we should be able to come up with a clever way to “smart follow” with a handy javascript snippet…

Building the Request for Replay

And a little bit of request fiddling as we figure out what headers are necessary we go from..

import requests
response = requests.post("https://tweepi.com/data/get_users_pp/follow_by_copy_paste.json",params={"userSnList":"foo","offset":0,"limit":25}, 
headers={
"X-Authorization": "ZnJlZTpncmlkLmZvbGxvd0J5Q29weVBhc3RlOjk0NjM0Mjg0MzYwMTI0ODI2MQ==",
"X-Requested-With": "angular",
"X-Tab-Id": "7166",
"Content-Type": "application/json;charset=utf-8",
"Accept-Language": "en-US,en;q=0.5",
"Accept": "application/json, text/plain, */*",
"Accept-Encoding": "gzip, deflate, br",
"Cookie": "c111990290-79992ic=c232338-43784-319745; c111990290-280953ic=c232338-43784-574953; tr2=1; tweepiapp=slqi3ldt8upm8oitf17eke44k2; kvcd=1514838972500; km_ai=G8OrOVIF0YFYftSnBsF7Qgi8aoM%3D; km_lv=x; km_vs=1"
})
response.content

to

import requests
response = requests.post("https://tweepi.com/data/get_users_pp/follow_by_copy_paste.json",params={"userSnList":"tonishabusch281","offset":0,"limit":20}, 
headers={
"X-Authorization": "ZnJlZTpncmlkLmZvbGxvd0J5Q29weVBhc3RlOjk0NjM0Mjg0MzYwMTI0ODI2MQ==",
"Cookie": "tr2=1; tweepiapp=slqi3ldt8upm8oitf17eke44k2;"
})
response.content


and we now can have some fun

Creating Twitter User Check List

The first thing is to get a list of potential users that we would want to follow. We can use the follower page from accounts that are well established. Since we don’t want to scroll for days let’s use a simple scroll function that we can turn off / control with the conditional. Once we have all the user nodes loaded we can run the second chunk of code to build a comma seperated user list that we will use in our python script; you’ll notice a copyToClipboard function at the end which allows us to easily select the entire list since console.log is disabled on Twitter and returns will be truncated.

// scroll until we see all users
var count = 0;
function st(){
  $("html, body").animate({ scrollTop: $(document).height() }, "fast");
  if(count < 2000) {
    count += 1;
    setTimeout(st, 500);
  }
}
st()
// then set count=999999

// XXX: done since console.log is taken over.. cute but not going to stop us
// create a div element and append twitter names to it (formatted for copy pasta in script above)
document.body.innerHTML += '<div id="userlist"></div>';
var eles = document.querySelectorAll('b.u-linkComplex-target');
for(var i=4; i<eles.length-5;i++) {
    document.getElementById('userlist').innerHTML += eles[i].innerHTML+',';
}

// console should print out the goodies when exiting the for loop but just in case
// never occured to me that something so simple would do the trick :D
// https://stackoverflow.com/questions/400212/how-do-i-copy-to-the-clipboard-in-javascript
function copyToClipboard(text) {
    window.prompt("Copy to clipboard: Ctrl+C, Enter", text);
}
copyToClipboard(document.getElementById('userlist').innerHTML)

and it works like a charm! 🎉

From there it’s time to use that user list inside a python script that will allow us to slam that endpoint as a free user. Since I don’t have a premium account there’s a limit of 25 users per bulk update, 500 for premium users.. maybe we can get around this somehow… but for now the script does the trick. In addition to updating the list we will most likely need to get a new valid tweepi session which will require us to change the tweepiapp value in the Cookie header.

import requests
import time
import json

def chunks(l, n):
    for i in range(0, len(l), n):
        yield l[i:i + n]

n = "INSERT_COPY_HERE"
names = n.split(',')        
split_names = list(chunks(names, 25))

count = 0
high_prop_users = [] # will use this to store "valuable" users
# XXX: why manage valid array index when you can try catch all the things... terrible terrible terrible but whatever im tired
try:
    while True:
        name_list = ','.join(str(x) for x in split_names[count])
        response = requests.post("https://tweepi.com/data/get_users_pp/follow_by_copy_paste.json",params={"userSnList": name_list,"offset":0,"limit":25}, 
        headers={
            "X-Authorization": "ZnJlZTpncmlkLmZvbGxvd0J5Q29weVBhc3RlOjk0NjM0Mjg0MzYwMTI0ODI2MQ==",
            "Cookie": "tr2=1; tweepiapp=7tk8us7lj933i743l4mos51q10;"
        })
        if '!doctype html' in str(response.content):
            print('TRY AGAIN')
        else:
            parsed_response = json.loads(response.content)
            users_info = parsed_response['users']
            for user in users_info:
                follow_ratio = user['followers_count']/user['friends_count']
                if follow_ratio > 0.8:
                    high_prop_users.append(user['screen_name'])
                    print('INSERTED')
            time.sleep(2)
        print('-----------------------------------------')
except:
    print(high_prop_users)
    exit(0)

Now that we have our final list of users we want to follow we just need another Javascript snippet to properly follow all those users.

Im sure there’s a better way to manage the parent search with jquery but it’s good enough for me / twitter since this is unlikely to change during the usefullness of this script (if it does it wont work and it’ll be an obvious fix). You may notice a strange id assignment at the beginning, this is to ensure that our simple for look can do something like lookup with getElementById and go from that base element to get anything else we might need..

// assign id to twitter handle for easy access using the list we built up
var names = document.querySelectorAll('.u-linkComplex-target')
for(var i=4;i<names.length-5;i++) {
  names[i].id = names[i].innerHTML.trimLeft(); // important to trim the random spacing in the div..
}

// follow everyone in our magic list
st = ['**', ..., '**']
for(var i=0;i<st.length-1;i++) {
    // sitting in console with a beer is definitely the best way to do this..
    $($(document.getElementById(st[i])).parent().parent().parent().parent().siblings()[1]).find('.user-actions-follow-button').click();
}

Using those 2 Javascript snipped and the hacky Python script we can succssfully re-create the functionality that Tweepi wants us to pay $9.99 a month for.

Playing the Twitter-game

I am not a marketer not do I have any real prior experience managing PR/social media for a company of any size. This is just a write-up of some of my learnings while out in the wild

By all accounts I am a Twitter novice; I joined a few years ago but don’t really keep up with it (rage tweet a politician or media personality from time to time but not much more). From other business ventures I’ve learned that having a strong presence on Twitter+Facebook can be a great way to drive traffic to your site and keep folks updated with any updates but I had never invested any time in growing the profiles (outside of word of mouth and the usual social links in the footer of a site). For my most recent project I decided to take an active role in the growth of my Twitter account and to attempt to use a few of these automation tools / API to make my life a little easier.

I started the journey by trying out a product called ManageFlitter.com; I had gone all in and decided to buy the smallest plan they offered to maximize my “strategic follows”. After about 2 days of bulk requesting it becamse obvious that the “snazzy dashboard” views were nothing more than a facade.. I was hitting rate limits and unable to process a single follow / request with the access grant I had enabled for the site.

At this point I started angrily emailing support to figure out why I was being blocked without hitting any of the actual API / request limits listed in the twitter guidelines. Here is the initial diagnosis I recieved (steps to fix are omitted since they were useless..)

Thank you for writing into ManageFlitter support on this. Unfortunately Twitter seems to have revoked some 3rd party app tokens over the last few weeks. This causes “invalid or expired token” in the notifications page and keeps your account from being able to process actions, post tweets into Twitter, or from allowing your ManageFlitter account to properly sync with Twitter.

Hmm so at this point I was frustrated because there is no way my token should have been revoked! Obviously they were using keys so that they could make these “analytic queries” on the twitter user base and had messed something up on their end that had made it impossible to proceed. I pressed along that line of thinking and received the following “helpful” response.

I am sorry to hear this. There seem to be a small group of Twitter users currently having this issue with ManageFlitter and some have noted that once their account is revoked, it is far easier for their account to continue to be revoked afterwards.

Some users have suggested that waiting 24 hours helps reset the account. Others have noted that the amount of actions they perform in sequence also matters greatly to avoid the revoked access. Some have noted that after they perform 100 clicks, if they wait 20-30 seconds (usually enough time to go to the Notifications page and see if Twitter is revoking access), then continuing to perform more clicks.

There is no particular reason why some accounts are being affected and other are not. We have been in touch with Twitter and unfortunately Twitter haven’t offered much in the way of remedy for us or our users.

TLDR; I was told to wait for the problem to fix itself..

This threw a massive wrench in my plans to bulk follow people inside the industry and hope that some percentage of them would follow me back. After a few more angry emails I was told to just wait it out.. At that moment I pulled out the classic "I will have to proceed with a claim to Paypal / the better business bureau" argument to get my money back and move on with another option.

After getting my money back I decided to ride the free train with Tweepi which had no problems for the first week of usage so I decided to buy a month of the lowest tier to use some of the analytics / follow tools that were being offered. With 2 weeks on the platform I can say that I’m very happy with what I paid for and will continue to use it in the future (until my follower count levels out a bit)

So why am I writing this article if I am just using a service to accomplish the task for me? While Tweepi does a lot for me it still imposes artificial limits on follow / unfollow in a 24 hour period. (see pic below)

You can see that the service has some limitations. The main one being that I can follow more people than I can unfollow in a given day. While that makes sense with Twitter’s policies my goal is a raw numbers game where I’d like to follow as many people as possible in hopes they follow me back. Whether they follow me back or not I am content to unfollow and continue with my following numbers game.

Through this process I was able to drive my followed count up quite a bit (considering my actual follower count)

but still had this problem of the unblanaced follow:followers that I wanted to correct. If I was active on Tweepi there was no way for me to drive this down without having to completely stop following people for a period while I unfollowed the max every day.

So today I decided to have a little fun inside the browser and see what I could do. 😀

Since twitter has a web + mobile application I could obviously sit and click through each of the people I was following to reduce the number but..

So let’s see how well formatted the twitter follower page is (and since it’s Twitter we know its going to be well organized). When arriving at the page we see the trusty un/follow button for each of the followers

we also notice that twitter has some infinite scroll magic going on to continuous load the 1000s of people we follow. With that knowledge in our hands it’s time to craft some Jquery flavored code to do the clicking for us

$.each($('.user-actions-follow-button'), function(value,index) {
  $(index).click();
});

Pretty easy to click through each of the buttons on the page but that’s only going to account for the ones we have manually scrolled through.. Not sufficient since we have >4000 but <20 buttons on the page. So let’s handle that damned auto-scrolling

var count = 0;
function st(){
  $("html, body").animate({ scrollTop: $(document).height() }, "fast");
  if(count < 2000) {
    count += 1;
    setTimeout(st, 500);
  }
}
st()

You might be thinking; why not just for loop this shit?! The scroll animation needs a bit of time to allow for the page load; if you call it too fast the entire page will bug out and the “click button” code wont work as expected. So we just use setTimeout and let that sucker run (good time to take a stretch or make some coffee). When you come back you should hopefully be at the bottom of the screen (wait for GridTimeline-footer to show up and you know you are done) 😀

Run the click code and patiently wait for your browser to slow down drastically and eventually unfollow your entire list. The result should look something like this

The 1 follower there through me off since when I clicked on the link for my followers there wasn’t anyone listed. At this point I was suspicious that I may have set off one of the limits that would have deactivated my accounts. I checked my emails and didn’t see any warnings or notifications from Twitter but did start seeing this whenever I tried to follow someone on their website. (Learn more here)

At this point I was thinking I just fucked myself and got my account banned or locked in some way. During this time of panic I decided to refresh my browser and saw some funky behavior on my profile page….

No “Following” count at all?! And I cant follow anyone because of some unknown policy breach..

After writing a short sob story to Twitter about how I had “accidentally unfollowed everyone”(cover my ass) I thought about the locking problem a bit more..Hmmm what about that Tweepi token I was using before? Who would have guessed that it would work and allow me to follow people again!

So with a little bit of crafted Javascript I was able to drop that Following count down without having to fight any artificial limits imposed on me by some third party. I’m incredibly happy with the results (as I am not banned and my account is working as expected) and plan to reproduce with another client in the future.

It’s always a good feeling when adding a new tool to the utility belt.