About Keepr 

Keepr is a tool for journalists to data mine breaking news in social web to get highlights as events unfold.

Mission statement:  Empower journalists to dig up a developing story in social media in real time efficiently--almost effortlessly.

prototype: http://www.keepr.com

source code: https://github.com/hqu/keepr

case study during Boston Bombing: "Twitter, Credibility and The Watertown Manhunt "



Feb 20, 2013

Had a meeting with  Ann Marie Lipinski, Joshua Benton, Chris Amico, Laura Amico, Ludovic Blecher, and Borja Echevarría de la Gándara.  Show them a demo of the simple video search mashup I built last week: keepr.com/video_wall.php

Got some great feedback for the future direction of this research project:

  1. Journalists have very different use cases over the course of their research
    1. get an overview summary of key highlights published as a story develops
    2. dig up early chatter of a breaking event
    3. follow a beat over a long period of time
    4. focus on a niche topic and get updates from that community of practice or subculture
  2. Keepr CANNOT attempt to serve the needs of all of these use cases.  It has to do one thing and do it really well.  What that will be is TBD.
  3. Journalists and technolgists should learn to speak the same language. The entire product development process requires close, effective collaboration.  This project can be a case study for how to work well together.
  4.  One method to quantify the "quality" of a tweet (or a media content embedded in the tweet) is to calculate a score for each individual message by pull the metadata that describes it, and assigning a factor weight to each of these descriptors.  For example:
    1. Tweet A has 10 retweets, the user has 50,000 followers, the video in the tweet has 100,000 views
    2. Tweet B has 100 retweets, the user has 10,000 followers, the video in the tweet has 5,000 views
    3. Score for tweet A = 10 + 50 + 100 = 160
    4. Score for tweet B = 100 + 10 + 5 = 115
    5. Of course this algorithm can be tweaked based on feedback from journalism on how helpful the resulting "quality" ranking is to them.
    6. Furthermore, we can expose some of weighing factors as controls in the user interface so that journalism can fine-tune the results by adjusting a set of sliders to get results that meet their use case needs as described in #1.

Feb 21, 2013

Done list

To Do List

Feb 22, 2013

To do List

explore front end frameworks

  twitter bootstrap


  Zurb Foundation


Feb 23, 2013

Done List

To Do List

Feb 27, 2013

I built a search functionality to bubble up conversations on Tumblr.  Surprisingly, the chatter there seems to be quite meaningful  and visually informative.


It is set to to search for "oscar".  

Also try to searching for "sequester".

For comparison, this page searches Twitter:


Mar 1, 2013

Mar 5, 2013

Done list

To Do List

Mar 6, 2013

Feedback from Nieman fellows

Mar 7, 2013

Done list

To Do list

Mar 8, 2013

Who are the users of Keepr?  And how 

To Do list

Mar 8, 2013

Testing at a live event at Harvard Kennedy School of Government

Lessons from the Forum: President of Senegal outlines three strategies for moving the country forward

Feature idea

Mar 11, 2013

Testing with SXWS search - worked

Mar 12, 2013

Mar 14, 2013

Testing with the new Pope Francis news  --> Worked!

The engine found the name of the newly elected Pope, country, his humble traits, etc...

Mar 15, 2013

Explore Pocket API

Extract links from Tweets

Examine Twitter linguistic patterns to reconstruct relationships and conversations


chron in php

Mar 16, 2013

Done list

To do list

Mar 17, 2013

Done list

To do list

Mar 18, 2013

metaphor: "Keepr: hire your personal Andy Carvin Twitter news desk in your mobile phone"

feature idea: automatically extract users, links, media, and conversations across a session and store it in a notepad

Feedback from Borja:

April 3, 2013

Done list

To do list

User test session

Feedback from casual user


Baseline comparison with Storify


April 4, 2013


User inputs a Twitter user account or Twitter List to seed the first search


Laura Amico demonstrated how she uses TweetDeck for Homocide Watch.  Her behavior inspires the metahpor of building Keepr to be an automated TweetDeck column generating tech--something users are familiar and comfortable with.

Wikipedia for News

Topic pages auto-generated Topic pages which then enables crowdsource editing of the links and stories.  Suspectible to gaming

News Aggregator

Starting with trending news topics on Twitter, crawl and index all the links and unpack them

Allow personalization based on username or Twitter List

APIs for any news org use the algorithm or display skin

White-label the service


User feedback

Cluster tags and tweets into story sections  

Enhance affordance to click on keyword buttons

Explain how the popular tweets are selected to be featured

De-dup tweets and keywords

Revamp alert feature to a permalink topic page

consider manually creating news category taxonomy.

April, 8 2013

To do list

feature request - "show me users who tweet good tweets"



April 9, 2013

Done list

April 11, 2013

Big ideas

Tip: PHP array_count_values function is the bestest, most useful NLP function ever!

Obstable - Twitter search only goes back a few weeks

Feature Request

April 15, 2013

April 18, 2013

Executive desicion to pivot

Josh Sterns’ process for source verification and his compilation of methdologies will guide future feature development for Keepr.  Period!

(This conclusion is partially inspired by my analysis of how the Boston Marathon breaking news coverage materialized.  And the competition is obvious already really good at spotting trending news, but hasn’t paid much attention to source credbility and follow up.)

New feature roadmap

Anyone in this hackpad has objections to the pivot?  Your thoughts are welcome.  Please type your thoughts below.


April 19, 2013

The manhunt in Watertown happened.  I used the method for tracking down twitter users as sources to get up to the second updates.  The result is this twitter list of reports on the scene:

I wrote an articles for Nieman Reports about source credbility

I did a radio interview with the Australian Broadcasting Company

April 25, 2013

Done list

To do list

Running into twitter’s rate limit again


April 26, 2013

Done list

Keepr now automatically expands search topics -- i.e. TweetDeck on autopilot is done!!

May 1, 2013

Done list


on mobile browsers, the tweets don’t display consistently

To do list

May 2, 2013

Journalism & the Boston Marathon Bombings:

Timing, Trust, and Credibility in the Age of Twitter



My presentation starts at 1:03:42 


May 18, 2013

Happy user quote:

June 22, 2013

Twitter updated it’s API and that broke Keepr.  

September 7, 2013

Keepr.com  is live and working

October 3, 2013

Twitter Lists of breaking events

November 15, 2013




Natural language search

Thoughts from Michael Slaby

Monitoring tool

Topic discovery

Impact analysis

Relative volume

Definite of what is "breaking news"

Advocay groups want to discover conversations and influencers

Trend over time - at what stage is the topic? emerging, peaking, ebbing

Contextual search



  Media type

  Breaking news

  Chatter -> Vetted

  Timeline news developments

    Query and sort by time


  task driven


  Save to virtual notepad




  Rolodex of sources

  Collaborative reporting

    research compilation

    Draft writing

  Predictive modeling of rumors to news



Personas and scenarios

  Zite user - news I need

  Newbie to social media

  General interest

  Social media editor

  Local reporter

  Beat reporter

Speculative => Developing => Vetted

How to incentivize users to participate?









Nice work!

 On Saturday, May 18, 2013, hackpad wrote:


 --- Burt Herman *Storify co-founder* burt@storify.com, +16504538787 *Winner, Online Journalism Award for digital innovation* http://storify.com