Ludum Dare 44 Retrospective

Jam Entry: https://ldjam.com/events/ludum-dare/44/wu-wei
HTML5 Playable Link: https://gamejolt.com/games/wu_wei/411422

Pre-theme Prep

I had been toying with the idea of taking on a level based platformer with a twist. Because of that I had gone and played a few platformer classics; Braid, Castlevania, Thomas is Alone, Super Meat Boy. Luckily the timing of playing these games lined up with a Humble Bundle for 8bit sprites and my personal desire to play with them and make some basic demos with the asset pack; I did this on stream and was chatting with folks about modern GM platform engines. Turned out that folks on Twitch were incredibly helpful and linked me to some real cool projects that I picked apart (Sonic clone was the main inspiration since I know historically it had a lot of great physic options)
About a month before the jam I had started to think about Unity / the possibility of doing something really uncomfortable in GM and trying to do a simple 3D game (2.5D dungeon crawler). Because of that thought I had started looking into level design history + tried a few level creations in Unity. This was an undeniably painful experience from the start but I definitely got better at turning a hand drawn mock into a playable level.. something that seemed impossible before and probably gave me the confidence to go the direction I went.

Theme Ideation

Your life is currency. My first thought is that it’s vague enough to allow for whatever we want to make. I’d decided previously that making something super “on theme” would not be my goal, I’d much rather complete something than try and fail to deliver something too ambitious in the 72 hour window. Rather than fight that fight I wanted to deliver a game that would be challenging for me personally to make. The game would be a platformer where your decisions to kill / spare the NPCs in the world would have consequences to your own life / progression in the game. The unique mechanic I wanted to try was to create a typical mobile game level system (selecting 1-N levels after unlocking) but introduce the element of decision permanence; if you kill a rat early on and return with a new ability he will be dead. The idea was to allow the casual to run through the game and the folks who were paying attention would enjoy new challenges and hidden areas.

Start Building the Engine

After getting the theme and deciding that I was going to run with a platformer I went ahead and started hammering out the basic mechanics. My philosophy for getting an engine going is; I want to spend as little time testing and the majority hammering out new mechanics to make the game fun. Because of that I just create a simple test room for my new obj_player and start to get the basics going; animation, states, input handling, physics, collisions.
quick testing room (at end of jam)
During this process we get animation + input handling first and of course have to deal with physics / interactions with world objects. Doing basic fall handling + collision checking is a fun adventure (drawing collision boxes is something I almost always do to ensure things are making sense); its really handy to add a conditional to your step function so you can increment step by step to ensure collisions are handled exactly like you want. In GM I’ll add a macro DRAW_COLLISIONS + this simple draw event (modifying for different objects as needed)
if DRAW_COLLISIONS {
    draw_set_alpha(0.3);
    draw_set_color(c_red);
    draw_rectangle(bbox_left,bbox_top,bbox_right,bbox_bottom,false);
    draw_set_alpha(1);
}
After getting floors + walls implemented I got spikes + other deal obstacles implemented. It was easy enough to throw together a test level to confirm we had a base that we could make a game around.. And then bedtime felt like it was well deserved

Day 2: Starting to Level Build

Since we end up writing code really late at night I like to revisit where things left off and do a bit of code cleanup + project organization. Nothing too crazy but unfortunately I had overlooked a few minor things that would result in a delay to get to actual level building / fun design. Normally this wouldn’t be a big deal but I had wanted to go to a BBQ and had to bail on it to because implementation couldn’t be done fast enough to make me comfortable with leaving level design to drunk self later in the afternoon. That decision was probably the reason I was able to bust out the 20 levels that I had set off to accomplish. Once the engine was “stable” level building took much much longer than expected + required me to learn a tool on the fly. Fortunately, Tiled is an incredibly nice program and was a breeze to learn on the fly. I think I had 5 levels completed with the basic run jump mechanics before deciding to create the menu system + the general GUI items. Since this is such an easy process I typically feel like its a nice win to get going into the final days of the compo.
coming together..

Mid Jam Disaster Recovery

Not necessarily the end of the world but something that I did not consider / think fully about.. Going from 8bit scale to 1920×1080 doesn’t seem like a big deal but the hardware needs to make everything happen seamlessly and it turns out scaling tilemaps is less than ideal and a known problem. GM:S 2 promises that this isnt an issue… although I tried to import my project and was still seeing the exact same tile breakage issue I was seeing in 1.4.X. In the article linked above there is a solve that involves making a new tilesheet that has bleedover on each tile to make sure scaling does not create a white/black spacing line. While there is a mention of how you’d go about doing that there wasn’t a “just use this script” type solution. Instead I found a link to a really handy program in the GM Forums. The program Tileset Champion does exactly what the blog post mentions; “Tileset Champion allows you to generate buffered tilesets from your existing tilesets”. While we weren’t able to just generate a new tilesheet and use that within the existing rooms. We gladly repainted all the rooms in Tiled and manually reimported the tile data into GML (recorded 2 vim macros; (1) pull from <tiles> section from exported room.gmx file and (2) replace <tiles> section in room GM generated). This allows us to keep all the object definitions that we had in the room and just have the tiles change.

Wrap Up

The final day was just a grind of levels and play testing. I had spent at least an hour the night before writing up what I wanted each level to accomplish / how I wanted to lay out the 20 levels with the mechanics that I had to play with. Because of that pen and paper work it removed a lot of the design decision making from the process and I could just focus on making levels that were playable and accomplished the goal of introducing / complicating a mechanic. I added music as the last touch and considered digging through the sound lib to find perfect jump / movement sounds but decided against it (mistake for my jam entry). At the end of the jam its hard to think rationally and I typically just want to hit submit and walk away from a computer for a long while. I submitted Wu-Wei and am proud of the base product and excited to push it forward into a complete title.
Posted in Development, GameMaker, Ludum Dare, Retrospective | Leave a comment

Simulate Mouse Click in GM 1.4

The goal here might sound pretty straightforward. Unfortunately, GM 1.4 does not provide anything directly out of the box that simulates a mouse click for that covers all implementations. It’s important to understand that in GM there are (almost always) multiple ways to achieve the exact same result. In the case of “Mouse Click” you can (1) use the explicit Event (Mouse Left Pressed/Released, etc) or (2) add a if mouse_check_button_pressed(mb_left) into the Step Event. A true mouse click simulation must ensure that the code we expect to invoke will be called when the function runs. For easy verification I created a button object that will show_message("Worked!") when it’s been clicked. I also defined another object with the same logic moved into the Mouse Left Pressed event so we ensure any method we come up will work in both scenarios. Assuming our function signature is simulate_left_click(x,y) we need to do 2 things; move the mouse to the coordinates and input a left mouse click at the current location. Luckily for us GML does provide the function window_mouse_set(x,y) which allows us to set the mouse location relative to the running game window. It works like a charm but we still need to figure out how to properly simulate a click. Intuition lead me to try event_perform but that will only handle 1 of the cases we are trying to solve for. If the mouse click logic is inside the Step Event it will never be triggered, so the solution won’t work for us. After some googling around it became clear that GML didn’t have anything in the language to do what we wanted. Since I am programming on Windows 10 I figured it would be worth exploring if we could do anything fancy in a dll to simulate a click that way (think game hacking / botting tricks). Fortunately we’ve been writing stupid hacks for old games lately and our Win32/DirectX research is going to come in handy! I won’t go into details about how to implement a dll for GM as there’s already plenty of information on the nets about that. First thing I tried was Win32’s SendInput as described here is alright for applications that aren’t running with DirectX.. For more information on why DirectX input vs Win32 see this SO post about simulating a mouse click in Warcraft3. The big idea is that since GM games are run with DirectX we must send DirectX input messages with SendMessage in order for anything to happen. We can actually checkout what messages are being send to the window while its being interacted with normally (image of messages seen by Spy++) From that we can see that we need to send WM_LBUTTONDOWNand WM_LBUTTONUP and everything should magically work.
extern "C" __declspec (dllexport) double __cdecl simulate_click() { 
HWND hWnd = GetForegroundWindow();
SendMessage(hWnd, WM_LBUTTONDOWN, 0, NULL);
SendMessage(hWnd, WM_LBUTTONUP, 0, NULL);
return 0;
}
And it does! We now import the dll and add the function call to our script and we have a true mouse simulation! For this being used in the wild see my Hobo Test Framework project
Posted in GameMaker, testing | Leave a comment

Creating GameMaker Extensions outside of GM Studio

I’ve been getting back into GM Studio 1/2 lately and wanted to revive a project I was working on. My first step was to move everything into an extension (rather than the shitty drag and drop method I had suggested before to use)
After going through the GM Creating Extension tutorial it was pretty clear I was not going to go through the process of adding a function within GM Studio (click Add Function in dropdown + fill out form)
For most simple extensions this is probably fine but I’m hoping to add and remove functions at will. Rather than managing the extension through GameMaker I’d like to be able to create the .gmez file myself and have it properly import into GM Studio 1.4.9999
By default a user would simply Right click the extension, click Export extension and give it an appropriate name. The created file will automatically get the .gmez extension and you can drag and drop the file into any project and it should work as expected.
Here’s an extension I exported from GM 1.4.9999 that has already been setup to open with WinRAR on my machine

Contents are as follows:
exported_extension.gmez:
-- Test.extension.gmx
-- Test/
   -- custom.gml
   -- Assets/
      -- Objects/
      -- ALL IMPORTED ASSETS CAN GO HERE (scripts, sprites, datafiles, etc)/
So you might be wondering how I already knew to open that with WinRAR.. I paid for an extension and it was mentioned in the docs that I could either drag and drop the extension and go through the normal flow of importing the assets object OR I could open the .gmez file with 7zip/winrar… the curious reader might be thinking this is just a .zip/.rar that is renamed.. you’d be close but let me show you the science I did to make importing my own extension work correctly.
Open up the example.gmez with your hex editor of choice and let’s look for some magic numbers! (I use WinHex. If you don’t have a hex editor its probably time to add one to the toolkit)
Since I’m not a noob to magic numbers I already have the List of file signatures purple on google and know a simple search of “37” will get me what I need.. and of course 37 7A BC AF 27 1C tells us we are looking at a 7-zip file
With some knowledge about how .gmx files are structured we can easily add additional functions and edit the .gml files outside of GM entirely! A simple test is to extract the .gmez file and create a default 7zip archive and drag and drop it into GM to confirm the results.
Since I’ve been modifying a lot of .gmx files on the fly with my GM Companion IDE doing the same for extensions feels like a natural feature add in the future. For now a simple scripted solution will do to tidy up a Github repo and hopefully allow others to contribute easier moving forward.
Random Note: I added a file to my extension, deleted it, and exported the extension. When I inspected the contents of the .gmez I noticed that the file that I inserted (and definitely deleted) was in the extension (massive .gml imported so was easy to notice).. Within GameMaker there was no reference to the file.. but I’m 100% it would be included in whatever is compiled.. possible to hide malicious code? (PoC || GTFO)
Posted in GameMaker, reverse engineering | Tagged | Leave a comment

Git Remote Annoyances on Windows

I use a few machines and just want them all to work. Problem is that never happens and I run into stupid things that keep popping up so… I’ll document it in case anyone else is getting frustrated by the same nonsense So you’ve got a new repo on github and you want to copy-pasta the commands to setup the remote+push.. thats alright until..
> git remote add origin git@github.com:DanBradbury/smashRecordKeeper.git
> git push origin master
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Only reason I knew what to do was because I banged my head against the wall to start off a Ludum Dare.. the issue here has to do with the current auth being in conjunction with git@github.com: linkage.. if we just use the https remote all will be fine.. pretty silly
> git remote add otro https://github.com/DanBradbury/smashRecordKeeper.git
> git push otro master
Counting objects: 290, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (286/286), done.
Writing objects: 100% (290/290), 2.59 MiB | 1.24 MiB/s, done.
Total 290 (delta 38), reused 0 (delta 0)
remote: Resolving deltas: 100% (38/38), done.
To https://github.com/DanBradbury/smashRecordKeeper.git
 * [new branch]      master -> master
and now we can git as expected on Windows w/ credential file auth (might popup in other cases.. but dont really want to test that stuff)
Posted in git, github, windows | Tagged , , | Leave a comment

Ludum Dare 43 Post Mortem

I’ll do my best to keep all my disappointment out of this post but I am obviously a little upset that after 27+ hours of work I didn’t have a game to deliver. I think there’s tons to take away from the entire experience even though technical tooling issues became too much to handle at the end.. Because after all that work I did make something that I was able to have some fun with.. even if I couldn’t share it like I was hoping to :/

What Ended Everything

I don’t think it’s fair to blame all my problems on a single extension but I will say that it was not smart on my part to try and use something I had never used full cycle (dev w/ git -> deliverable .exe / HTML package).. I’m usually not the one to buy into the marketing hype but I was sold as soon as I saw this gif
I saw that late one night and ended up pulling the trigger right before the Jam started.. Not testing the tool (GMLive) adequately before jumping into a competition was a mistake.. should have / could have used a day before the Jam to create a small project that I had tested the creation of an .exe + HTML package In the end I’ve created a post on the itch.io page for the GMLive project in case my issue might be something larger (can contribute to the project) / obvious that I shouldn’tve done during development. After 3 days in the weeds, hunting down a bug that only exists when the extension I was using was turned off.. was enough for me to throw in the towel with < 3 hours left (I’m editing this post a day after the event and have spend 1-2 hours doing further debugging without any luck.. will get it eventually…1 week later still haven’t had the heart to hunt it down)

Theme Ideation

Had an idea pretty early on that a tower climb card game w/ small deck size + hard choices after each level (choose 1/3 cards to add to your deck but others are given to next level boss) would be something that I’d like to build. Main “interesting” gameplay mechanic was cards you passed on (being used be subsequent enemies) would get upgraded during harder levels making your decisions to “sacrifice cards to the tower” that much harder and more important during the actual gameplay.
Personal goal was to deliver 4/5 levels of increasing difficulty that would build/introduce card mechanics.. playing MTG since I was little this felt like something I wanted to take on and was up for (no matter the challenges of balance, design, etc)
Overall I was pleased with the theme and happy that I came up with an idea and stuck to it (still believe gameplay/general idea would have been enjoyable)

Wasted Time

Looking back at the time-lapse video + commits throughout the event its pretty clear that I got ahead of myself on the art side of the project. I don’t think it was a smart call to start messing with any artistic tweaks before multiple levels were built + tested in .exe/HTML versions (always problems). Of course hindsight is 20-20 but the LD before my mantra was “Don t waste time play testing or trying to be an artist”. Unfortunately I think I did both.. too excited about making a card game + distracted by the fact I made something pretty on accident Should honestly write the mantra on a whiteboard in my room before the Jam starts.. creating cool looking shit is great but that’s really not me.. should get help on that front in the future if I want things to look pretty..

Custom IDE Experience

I’ve been working on a custom IDE / GM Studio companion application to allow me to edit / compile my projects outside of GM. I’d say that 95% of the time I was using my IDE and the other 5% was spent dealing with GM Studio complaining about memory access issues + strangeness with accidentally creating 2 scripts with the same name or having an object deleted but project.gmx file not being updated I will definitely be making stylistic changes to the IDE (was not built for multiple resolution so using on a laptop was pretty silly) to increase usability + functionality when editing larger projects; had initially shrugged off implementing proper TreeView structure for objects/scripts in “Groups” since the directory structure was “good enough”.. turns out it would have been nice to use and abuse groups and the project grew.. IDE did not handle that and was not going to swap to C# mid Jam and implement XML parsing In the end the ghetto solution I’ve got pre-Github roll up is workable and was enjoyable to use during the Jam but is very much incomplete in terms of the full fledged experience I was hoping for. Big things to knockout in the future are; (1) changes in IDE look the same as changes in GM (num spaces / tabs?), (2) usability changes for viewing in different resolutions / allow the frame to be adjusted as needed (found myself just wanting to click through scripts and read contents within the IDE.. dynamic “Inspect object” section might be nice), (3) Build tree structure from project.gmx file.. this is all GM Studio 1.4.9999 seems to care about so we should do the same (crawling directories was nice + worked but… would avoid lots of headache if the content of that was guaranteed + confirmed by IDE rather than having to open GM to complain about something)

Engine Building

I felt slow during this phase of the compo. Think I should have focused on 2/3 cards only early on and created the basics for gameplay on multiple levels ASAP. When I went to bed not competing the Initial Battle phase that was written on my whiteboard I think I was in a bad spot. Not a terrible spot but definitely not where I should have been (especially if I was on a team.. cant imagine showing up in the morning with a half working engine that looked like shit).. Should have pushed harder for that base engine / been more focused on limiting the scope of the MVP Because of the things that were taken on unnecessarily early, I think that the battle logic code is all over the place.. right now I’m afraid to look at the code but am certain the logic for battling morphed and morphed into something that was going to be cleanly handled by 1 object into 2 objects juggling responsibilities inside of a STEP function that was probably calling multiple alarms.. Maybe its not that bad… but I think code review post mortem would be a separate post entirely

Takeaway

LDs are awesome and a great way to test yourself! I’m glad I pushed myself to build something that made me uncomfortable (felt like a much larger ask than the text / platform games I usually throw together for these things). While I didn’t turn anything in I’m still sitting on a codebase that I’m hoping will be workable and understandable when I revisit it with a clearer head. I’ll definitely be competing in the next Ludum Dare and will put it on my calendar now so I don’t find out about it a few days before..

Links

Posted in tooling | Tagged , | Leave a comment

Git Auth when Windows Credentials are Locked / Just want to use file

Most Windows users won’t run into this problem because they don’t have a system administrator controlling their machine but for anyone who is experiencing weirdness while using wincred here’s a brief explanation of the problem + an easy fix to get things back up and running.

The wincred issue that I was facing was anytime I ran powershell as a non-administrator I would be prompted for my Username/Password and rather than it being persisted I would get a fun CredWrite failed error and would be prompted again on each subsequent git related call. At first I was at a total loss and was running everything as an admin just to avoid that annoying prompt when trying to pull/push/etc. I finally decided enough is enough and started doing some digging on how windows creds were being persisted on my machine.

Upon looking up the CredWrite failure it became clear that this was a Windows specific Git credential manager that I was oblivious to. The default storage on Windows is something called wincred, which is essentially a Windows built-in credential manager that can be accessed via Control Panel

Unfortunately for me when I drill into the manage page I get the following

Welp that explains the CredWrite failure I was seeing all over the place. Since I was unable to use the recommended a git-credential-store helper I had to resort to using the store –file option to get things working as I’d expect on any other machine (less than ideal but worth it to not be forced to type my username/password every time I wanted to git)

There are a few options to disable the Git Credential Manager (detailed here) but here’s the method I used to get unblocked:

  • git config –global credential.helper “store –file ~/.gitcredentials”
  • Create gitcredentials file and place my creds in there according to the specified format
username=myuser
password=mypass
  • git config –edit –system
  • Remove the helper = manager line that is still in the system even though we specified to use the file for cred info
  • Celebrate because we can git without being prompted every time!

The main issue I had was that I thought setting the file store in (1) would be enough but still ran into the problem where the cred manager was still being used. Checking the global config is critical to ensure that things are going to work as expected and it’s not a bad idea to know exactly what’s going on at the global level

Posted in git, windows | Leave a comment

Twitter Gaming: Tweepi.com Functionality Rip

There are plenty of twitter management tools out that will help automate / manage your twitter account for a few bucks a month.

While most products don’t add too much value I’ve found some particular features on Tweepi.com that have been pretty helpful. In particular there’s a stat called Follower Ratio which lets you know how likely a user is to follow you back; > 80% is a user thats probably worth following (and unfollowing later once we have a healthy count..will lose some followers from bot automation but majority of real users won’t unfollow back)

When using the free version of the site there is a tool that allows us to search up to 25 users using a comma seperated list. This definitely looks like a tasty target and upon checking the network tab we can see a clean POST with easily understood params and response. A test response is shown below

We can see from the response below that calculating Follower Ratio is as easy as followers_count/friends_count is all we need to process a user object in our script.

POST https://tweepi.com/data/get_users_pp/follow_by_copy_paste.json
....

RESPONSE:
{
   "total":1,
   "page_size":20,
   "users":[
      {
         "user_id":"*****",
         "screen_name":"*************",
         "location":"",
         "full_name":"*************",
         "last_tweet_time":"1969-12-31 16:00:00 -0800", // cute
         "followers_count":1,
         "friends_count":89,
         "statuses_count":0,
         "profile_img_url":"\/\/abs.twimg.com\/sticky\/default_profile_images\/default_profile_normal.png",
         "is_verified":false,
         "utc_offset":null,
         "bio":"it's hi astonishing ocean blue eyes that iadore the most but it's his contagious bright smile that transforms my orrowful frown into something joyous",
         "url":"",
         "lang":"en",
         "is_protected":false,
         "member_since":"2017-09-06 05:02:47 -0700",
         "listed_count":0,
         "favourites_count":0,
         "default_profile":true,
         "default_profile_image":true,
         "geo_enabled":false,
         "ui_last_updated":"2018-01-01 12:36:23 -0800",
         "is_follower":false,
         "is_friend":false,
         "follower_or_following":" ",
         "history_date":"2000-01-01 00:00:00 +0000",
         "is_safelisted":false,
         "tag_names":[

         ],
         "tag_dates":[

         ]
      }
   ]
}

As long as we can replay this request then we should be able to come up with a clever way to “smart follow” with a handy javascript snippet…

Building the Request for Replay

And a little bit of request fiddling as we figure out what headers are necessary we go from..

import requests
response = requests.post("https://tweepi.com/data/get_users_pp/follow_by_copy_paste.json",params={"userSnList":"foo","offset":0,"limit":25}, 
headers={
"X-Authorization": "ZnJlZTpncmlkLmZvbGxvd0J5Q29weVBhc3RlOjk0NjM0Mjg0MzYwMTI0ODI2MQ==",
"X-Requested-With": "angular",
"X-Tab-Id": "7166",
"Content-Type": "application/json;charset=utf-8",
"Accept-Language": "en-US,en;q=0.5",
"Accept": "application/json, text/plain, */*",
"Accept-Encoding": "gzip, deflate, br",
"Cookie": "c111990290-79992ic=c232338-43784-319745; c111990290-280953ic=c232338-43784-574953; tr2=1; tweepiapp=slqi3ldt8upm8oitf17eke44k2; kvcd=1514838972500; km_ai=G8OrOVIF0YFYftSnBsF7Qgi8aoM%3D; km_lv=x; km_vs=1"
})
response.content

to

import requests
response = requests.post("https://tweepi.com/data/get_users_pp/follow_by_copy_paste.json",params={"userSnList":"tonishabusch281","offset":0,"limit":20}, 
headers={
"X-Authorization": "ZnJlZTpncmlkLmZvbGxvd0J5Q29weVBhc3RlOjk0NjM0Mjg0MzYwMTI0ODI2MQ==",
"Cookie": "tr2=1; tweepiapp=slqi3ldt8upm8oitf17eke44k2;"
})
response.content


and we now can have some fun

Creating Twitter User Check List

The first thing is to get a list of potential users that we would want to follow. We can use the follower page from accounts that are well established. Since we don’t want to scroll for days let’s use a simple scroll function that we can turn off / control with the conditional. Once we have all the user nodes loaded we can run the second chunk of code to build a comma seperated user list that we will use in our python script; you’ll notice a copyToClipboard function at the end which allows us to easily select the entire list since console.log is disabled on Twitter and returns will be truncated.

// scroll until we see all users
var count = 0;
function st(){
  $("html, body").animate({ scrollTop: $(document).height() }, "fast");
  if(count < 2000) {
    count += 1;
    setTimeout(st, 500);
  }
}
st()
// then set count=999999

// XXX: done since console.log is taken over.. cute but not going to stop us
// create a div element and append twitter names to it (formatted for copy pasta in script above)
document.body.innerHTML += '<div id="userlist"></div>';
var eles = document.querySelectorAll('b.u-linkComplex-target');
for(var i=4; i<eles.length-5;i++) {
    document.getElementById('userlist').innerHTML += eles[i].innerHTML+',';
}

// console should print out the goodies when exiting the for loop but just in case
// never occured to me that something so simple would do the trick :D
// https://stackoverflow.com/questions/400212/how-do-i-copy-to-the-clipboard-in-javascript
function copyToClipboard(text) {
    window.prompt("Copy to clipboard: Ctrl+C, Enter", text);
}
copyToClipboard(document.getElementById('userlist').innerHTML)

and it works like a charm! 🎉

From there it’s time to use that user list inside a python script that will allow us to slam that endpoint as a free user. Since I don’t have a premium account there’s a limit of 25 users per bulk update, 500 for premium users.. maybe we can get around this somehow… but for now the script does the trick. In addition to updating the list we will most likely need to get a new valid tweepi session which will require us to change the tweepiapp value in the Cookie header.

import requests
import time
import json

def chunks(l, n):
    for i in range(0, len(l), n):
        yield l[i:i + n]

n = "INSERT_COPY_HERE"
names = n.split(',')        
split_names = list(chunks(names, 25))

count = 0
high_prop_users = [] # will use this to store "valuable" users
# XXX: why manage valid array index when you can try catch all the things... terrible terrible terrible but whatever im tired
try:
    while True:
        name_list = ','.join(str(x) for x in split_names[count])
        response = requests.post("https://tweepi.com/data/get_users_pp/follow_by_copy_paste.json",params={"userSnList": name_list,"offset":0,"limit":25}, 
        headers={
            "X-Authorization": "ZnJlZTpncmlkLmZvbGxvd0J5Q29weVBhc3RlOjk0NjM0Mjg0MzYwMTI0ODI2MQ==",
            "Cookie": "tr2=1; tweepiapp=7tk8us7lj933i743l4mos51q10;"
        })
        if '!doctype html' in str(response.content):
            print('TRY AGAIN')
        else:
            parsed_response = json.loads(response.content)
            users_info = parsed_response['users']
            for user in users_info:
                follow_ratio = user['followers_count']/user['friends_count']
                if follow_ratio > 0.8:
                    high_prop_users.append(user['screen_name'])
                    print('INSERTED')
            time.sleep(2)
        print('-----------------------------------------')
except:
    print(high_prop_users)
    exit(0)

Now that we have our final list of users we want to follow we just need another Javascript snippet to properly follow all those users.

Im sure there’s a better way to manage the parent search with jquery but it’s good enough for me / twitter since this is unlikely to change during the usefullness of this script (if it does it wont work and it’ll be an obvious fix). You may notice a strange id assignment at the beginning, this is to ensure that our simple for look can do something like lookup with getElementById and go from that base element to get anything else we might need..

// assign id to twitter handle for easy access using the list we built up
var names = document.querySelectorAll('.u-linkComplex-target')
for(var i=4;i<names.length-5;i++) {
  names[i].id = names[i].innerHTML.trimLeft(); // important to trim the random spacing in the div..
}

// follow everyone in our magic list
st = ['**', ..., '**']
for(var i=0;i<st.length-1;i++) {
    // sitting in console with a beer is definitely the best way to do this..
    $($(document.getElementById(st[i])).parent().parent().parent().parent().siblings()[1]).find('.user-actions-follow-button').click();
}

Using those 2 Javascript snipped and the hacky Python script we can succssfully re-create the functionality that Tweepi wants us to pay $9.99 a month for.

Posted in python, tweepi, twitter | Leave a comment

Bioshock: Infinite Review


The initial introduction to the underwater dystopian society of Rapture in the original Bioshock was something that I recommend every real gamer experience (even if you aren’t an FPS kind of person). The familiar themes of authoritarian rule and dark despair from the previous 2 installments make a return and should be a pleasant reminder of the previous roller coaster rides we’ve taken before.. Nostalgically remembering Atlas… if you don’t get that you should seriously sit down and play the first game (wait for Steam sale and buy all 3 games w/ remastered versions for < $15)

Alright so it’s probably not fair to assume that everyone has played the previous games but luckily the majority of the gameplay doesn’t require any previous series knowledge to enjoy. The change from the enclosed spaces of Rapture to the limitless freedom of the skies of Columbia is a welcome and creative change. The introduction of the Skyhook is the main way to harness some of that open space and feels great when it can be properly utilized in a consequential battle. Unfortunately for all the fun the new mechanics introduced they do get stale pretty quickly and for the most part are completely unnecessary to accomplishing the task of dispatching waves of baddies.. (1)Flying around on a Skyhook loop while you pick off the hordes of enemies slowly without taking a scary amount of damage feels lame just to move on to the next segment. (2) Having Elizabeth’s animation to throw you Ammo/Salts/Health during battle felt amazing the first few times but after learning to exploit the throws; wait till low ammo for toss to maximize shooting time + know she can throw that thing from anywhere (she will teleport for you so don’t stress). For 80% of the game you won’t run into these annoyances (resources are good, no real advantage on Skyhook, no buggy Elizabeth) but it cannot be ignored or missed in a playthrough (maybe I’m the only one that was bothered by this on both my runs of the game).

For the most part the sequenced fights are well planned and force the player to think about cover and how they will use their various Vigors to overcome the reasonable challenge. There are other times when the “ambush” is obviously planned and it feels like you are just doing a dance to kill 3 waves before you can open the next door to do the same thing. While predictable fights and AI funnelling does occur from time to time these occasions are in the minority to the meaningful fights where a pack of Patriots shows up and presents a welcome last annoyance before you can get on a gondola to the next section of Columbia. Combat consists of a combination of using Vigors and guns to dispatch of enemies in whichever way you see fit; I ignored the pistol and leveled up my shotgun, machine guns and sniper while maxing out salts and abusing crows, soul steal and lightning on machines. The option to take combat in whichever direction you want with various Vigor upgrades along with Armor, Health, and Salt upgrades found along the way is well executed and is one of the main reason one would come back for a second / third playthrough. The one gripe I have with the RPG-lite-esque combat system is that the entire clothing upgrades felt entirely pointless to change along the way, juggling the various outfits that gave additional combat effects seemed really awesome but in reality it feels like a cumbersome addition that isn’t even worth paying attention to.

The storyline follows along the lines of a dystopian society that hails a demigod and his family. Playing as Booker DeWitt we are responsible for rescuing the princess from her captures at all costs. Without giving anything away (just in case) our damsel in distress is a young girl named Elizabeth who can open rifts into other dimensions, justifying her isolation and protection by a massive songbird. As you can imagine the story plays with themes of freedom and imprisonment, which is particularly interesting given the idea that Columbia has seceded from the United States right after the abolition and obviously wasn’t too happy with that Lincoln guy. While the story is nothing spectacular it delivers an engaging world that feels worth exploring and trying to better understand through the audio diaries scattered.

In reality when the game is at its best (the majority of the time) it shines bright but there always seems to be some lingering blemish that prevents the game from being a true masterpiece. While I wanted to get lost in the beauty and creativity of the circa 1900 American secessionist city of Columbia, the game constantly trips over itself with the inconsequential mechanic or monotonous fight that tend to feel like nothing more than filler.

Overall a worthwhile experience with a few hiccups that prevent it from being a true classic on its own. Expect a remastered version to be released in a few years.

4/5

Posted in Game Review | Leave a comment

Reverse Engineering ‘Product Catalog’

Recently I’ve been looking at an application that has some data that I’d like to scrape and use/format for my own selfish desires. The application we are targeting is on iOS + Android so I went through the usual flow on my iPhone
– mitmproxy – no good. traffic must be SSL / non-HTTP. Android cert-pinning stops app on init
– wireshark w/ rvictl on iOS device – SSL traffic pulling initial data catalog / no API fetches once app is initialized (sqlite init)

After getting blocked and giving up for about a month I decided to root my OG Nvidia Shield and dig a little deeper..

Digging Deeper

Prereqs

Root phone -> Install Xposed Framework -> Download [Inspeckage] Module -> Get Started

My initial hope was that Inspeckage would solve all my problems and I wouldnt have to dig too much into Xposed but in the ended up being the launching pad to the golden catalog.

After turning on Inspeckage and letting the app boot up for the first time we can see a few things. (1) https traffic, (2) files created zip/certs/js+img assets. I was hoping the SQLite tab would light up and we could query against .db file immediately but that would be too easy

Ripping things apart

Before getting into the .apk I figured it would be worthwhile to see what was stored on my device
adb shell && cd /data/data/com.package.name/
and… explore!

The main thing I was looking for here was something in the databases/ or files/ directories. Luckily we were able to spot a TARGET.db within files/databases but it was definitely encrypted (dreaded “file is encrypted or not a database” when querying)

Remember that if the app is not debuggable you won’t be able to adb pull /pathtoDBFile.db instead you’ll have to adb shell && su && mkdir /sdcard/data && cp /pathoDBFile /sdcard/data and then pull that file

Inspeckage has the handy download .apk so there’s a few things we can do here (starting with the most obvious)
– unzip it and take a look at what we’ve got

dan@dan-MacBookPro:~/riperino$ unzip og.apk
Archive:  og.apk
  inflating: META-INF/MANIFEST.MF
  inflating: META-INF/CATEXTKY.SF
  inflating: META-INF/CATEXTKY.RSA
  inflating: AndroidManifest.xml
  inflating: assets/_where-is-www.txt
  ...
 extracting: assets/icudt46l.zip
 extracting: assets/www/assets/
 .... a ton of assets (selected some potential highlights)
  inflating: assets/www/config/config.json
  inflating: assets/www/cordova.js
  inflating: assets/www/css/bootstrap/
  inflating: assets/www/js/
 extracting: res/drawable/icon.png
  inflating: res/xml/config.xml
 extracting: resources.arsc
  inflating: classes.dex
...
  inflating: lib/armeabi/libsqlcipher.so
  inflating: lib/armeabi-v7a/libsqlcipher.so
  inflating: lib/x86/libsqlcipher.so

From looking at the contents we can tell the basic structure but we are mostly interested in a few files
– libsqlcipher.so & assets/icudt46l.zip (interesting fallback zip for sqlcipher..possible fun attack vector)
– classes.dex (exactly what it says)

This confirms our hunch about SQLite and explains the missing contents from the SQLite tab while using Inspeckage. Now the hope is that classes.dex reveals something blatantly obvious..

Reading some code

There are quite a few tools for digging into the contents of that .dex file but I’ll go over my comfortable flow (unnecessary .dex -> .jar step with tools like Bytecode Viewer but the extra step does provide some extra possibilities if we really need to build a custom apk)

Using the dex2jar toolkit we can use the classic d2j-dex2jar to give us the handy .class files zipped to .jar (while not perfect this is almost always “good enough” for what we need)

After creating the .jar its time to pop open Java Decompiler and dig into the package in question.. And after a little while we run into this section of code (masterKey makes us happy and definitely worth hooking into?)

After reading the docs for getReadableDatabase it’s clear that’s indeed the SQLite password and if the developer also followed the documentation they would also have called createAndGetDBPath which would serve as the initialization.. Lo and behold the DBHelper.class implements that class and we can add a hook to ensure we catch the key when we reboot the app

Hooking and Winning

Without getting too tangential Xposed is great and the documentation is outstanding for anyone interesting in getting started. If the development tutorial is not enough the content online was more than enough to clear up any potential blockers.

public class Main implements IXposedHookLoadPackage {
    public void handleLoadPackage(final LoadPackageParam lpparam) throws Throwable {
        findAndHookMethod("com.****.core.db.DBHelper", lpparam.classLoader, "createAndGetDBPath", Context.class, String.class, new XC_MethodHook() {
            @Override
            protected void beforeHookedMethod(MethodHookParam param) throws Throwable {
                XposedBridge.log("Hooked em': DBHelper#createAndGetDBPath ");
                XposedBridge.log("----------------------------------------");
                XposedBridge.log(param.args[1].toString());
                XposedBridge.log("----------------------------------------");
            }
        });
    }
}

We load our hook module, reboot our device and when we rerun the app we see something beautiful in the logs

Hooked em': DBHelper#createAndGetDBPath
----------------------------------------
base64masterKey== (totally legit..)
----------------------------------------

Now I should be able to

Sqlcipher rippedDB.db
PRAGMA key=”base64masterKey==”
.tables

And get something other than file is encrypted or not a database. Unfortunately this didn’t work on my machine and I thought I was getting juked out and went to bed.

After thinking about questions like these (1, 2, 3) all night + the following day of family time it became clear versions must be the issue in my case. After figuring out the difference between Commercial and Community it became clear this was just a time save and the same version was being used for both and the commercial folks were just given the latest binaries without having to work for them. Since we have some compatriots who dont like working much either I ran brew install sqlcipher and of course things just worked..

Mac - 3.15.2
Ubuntu 16.04 - 3.8.6

As an additional aside there are some open source “sqlbrowser applications” that have sqlcipher as a feature but dont do a great job revealing the versioning. Just build community versions as they are released and things should be fine / bank on everyone being on commercial and brew updating at the same time 😀

Upon using the correct version we see a beautiful table list and can query to our heart’s content.

Retro

The entire sqlcipher seems incredibly trivial given the release strategy and knowing what we know about hooking. The fact that the key is going to be passed around no matter what makes things quite easy unless some additional precautions are taken but unfortunately it just moves the problem a step deeper. In the end the Android SQLDatabase methods are going to used and we can hook them no matter how many Proxys/Helpers are introduced to the codebase. After digging further into the Xposed modules someone wrote a generalized hook that looks for the specific sqlcipher native package to hook into.. This seems to reinforce my belief that it’s general uselessness but there must be a reason big companies are giving them money (other than the desire to avoid building it themselves)

Posted in reverse engineering | Leave a comment

Ixalan Draft #1

Black/Blue/White Deck

Didnt realize I messed up mana until I got home.. would have preferred a 664

  • 7 Swamp
  • 6 Island
  • 3 Plains
  • Ixalans Binding
  • Slash of Talons
  • Cancel
  • Spell Pierce
  • 3 One With the Wind
  • 2 Run Aground
  • 2 Depths of Desire
  • Duress
  • 2 Call to the Feast
  • Deadeye Plunderers
  • Anointed Deacon
  • Desperate Castaways
  • Wanted Scoundrels
  • Seekers Squire
  • Kinjallis Caller
  • Shining Aerosaur
  • Paladin of the Bloodstained
  • Prosperous Pirates
  • Sailor of Means
  • Shipwreck Looter

sideboard

  • Charging Monstrosaur
  • Dire Fleet Captain
  • Kinjallis Caller
  • Gilded Sentinel
  • Lookouts Dispersal
  • Queens Commission

This is the deck that I boxed up when I headed home.. Looking at it now I can see why I lost my last 2 matches

In my first game I ran Charging Monstrosaur (swapped after game for another One With the Wind since it won my the first game..not the best idea to run 3) and the second Kinjallis Caller which was played on turn 1 and proved to be an annoyance for my opponent (swapped for a second Call to the Feast.. not a bad card but wasnt necessary)

I ended up winning both games with a timely Ixalans Binding and One With the Wind which led me into a false sense of security for my next 2 matches (both of which I think I should have won)

Posted in Friday Night Magic, MTG | Leave a comment