https://pixelastic.github.io/marvel/

Marvel Super-Search!

Tim Carry

Getting the data

Wikipedia

Too many sources

Hard to use:

  • Raw results
  • Custom markup
  • Not structured
  • Unofficial projects
  • Frozen in time
  • Incomplete

wiki-infobox

wikidata-sdk

x-ray

HTML scraping

Marvel API

http://gateway.marvel.com/v1/public/characters/1009262

2 years old

Unreliable:

  • Timeouts
  • Infinite loops
  • Empty results
  • Slow
  • Rate limit

retryUntilItWorks()

Marvel logo

Daredevil thumbnail

Abandoned by his mother, Matt Murdock was raised by his father, boxer "Battling Jack" Murdock […]

827 comics, 1326 stories

Marvel Website

http://marvel.com/characters/11/daredevil

Different than API

Manual scraping again

Awesome for design

#3F0606

Isolated scripts

One script per source

 

Scraping

  • Easy
  • Slow
  • Temporary files

Parsing

  • Complex
  • Fast
  • Commited in git
$ npm run dbpedia
./download/dbpedia
  ├── 8-Ball_(comics).json
  ├── Abdul_Alhazred_(comics).json
  ├── Abigail_Brand.json
  […]
  ├── Zombie_(comics).json
  ├── Zom.json
  └── Zzzax.json

Building the UI

library of UI widgets

fully customizable

eat your own dog food

10k records free

Facet screenshot

<h3>Teams</h3>
<div id="teams"></div>
[…].refinementList({
  container: '#teams',
  attributeName: 'teams',
  operator: 'and',
  limit: 10
})
.ais-refinement-list--label {
  cursor: pointer;
  font-weight: normal;
}

Cloudinary

http://res.cloudinary.com/pixelastic-marvel/image/fetch/ w_450,q_90,e_colorize:40,co_rgb:3F0606/ http://i.marvel.com/i/03/537ba78541492.gif

Image CDN

Resize and compress

Tons of effects

7.5k operations free

Deadpool comic panels

Tainted Deadpool comic panels

tim@
.com