Super-Powers with APIs
Tim Carry
@pixelastic
Getting the data
Two sources
Wikipedia: Data Quantity
2000+ characters
Wikipedia API
https://en.wikipedia.org/w/api.php?
titles
=
Captain_America
&
format
=
json
&
action
=
query
&
prop
=
revisions
&
rvprop
=
content
Wikipedia API
{{Other uses}} {{About|Steve Rogers|the subsequent versions of the character|List of incarnations of Captain America}} {{pp-vandalism|expiry=03:36, 23 May 2016|small=yes}} {{Infobox comics character | image = CaptainAmerica109.jpg | converted = y | caption = ''Captain America'' #109 (Jan. 1969).Cover art by [[Jack Kirby]] and [[Syd Shores]]. | alt = Captain America bursting […]
  • Raw data dump
  • Custom markup
  • Not practical
DBPedia
http://dbpedia.org/data/
Thor_(Marvel_Comics)
.json
{
 "abstract": [{
   "lang": "en",
   "value": "Thor is a fictional character, a superhero […]"
 }],
 "aliases": [{
    "lang": "en",
    "value": "Dr. Donald Blake, Jake Olson, Sigurd Jarlson, Eric Masterson"
 }]
}
  • Structured data
  • Community project
  • Frozen in time (April 2016)
Marvel API: Data Quality
http://gateway.marvel.com/v1/public/characters/
1009351
{
"name": "Hulk",
"thumbnail": "www.marvel.com/538615ca33ab0.jpg",
"description": 
  "Caught in a gamma bomb explosion while trying to save the life of a teenager, [...]",
}
  • In-universe description
  • Cohesive thumbnails
  • Character popularity
Marvel API: Worst practices
Slow
Rate limit
Timeouts
Infinite redirects
Empty results
200 OK
errors
Marvel Website
http://marvel.com/characters/
25
/
Hulk
  • Not using the API
  • Custom background
  • Custom color
Web Scraping: Poor man's API
<div class="description">
  <h4>Hulk</h4>
  <img src="./hulk.png" class="thumbnail"/>
</div>
$('.description h4').text()
$('.description img').attr('src')
{
  "name": "Hulk",
  "thumbnail": "./hulk.png"
}
  • Nokogiri, Scrapy, Cheerio
  • Always available
  • Breaks often
Two-step process
1
Download
  • Slow
  • Easy
2
Parse
  • Fast
  • Hard
    TDD
Several sources
  • Isolate scripts
  • Merge data
  • Define fallbacks
  • Commit in git
Building the UI
Search:
  • Push JSON
  • UI widgets
  • HTML / JS / CSS
Images:
Cloudinary
http://res.cloudinary.com/{app_id}/image/fetch/
w_450
,
q_90
,
e_colorize:40
,
co_rg:17250B
/
http://i.marvel.com/i/537bafb19e9f4.gif
Hosting:
GitHub Pages
git push origin gh-pages
https://{username}.github.io/{reponame}
  • Git
  • Free
  • Data is everywhere
  • Tools are available
  • No servers are required
Go and build stuff!
community.algolia.com/marvel-search
Super-powers with APIs
Tim Carry
Super-powers with APIs
Tim Carry