Skip to content

Instantly share code, notes, and snippets.

@travisbrown
Last active August 29, 2015 13:57
Show Gist options
  • Save travisbrown/9609756 to your computer and use it in GitHub Desktop.
Save travisbrown/9609756 to your computer and use it in GitHub Desktop.
scalaVersion := "2.10.3"
resolvers ++= Seq(
"Scalaz Bintray Repo" at "http://dl.bintray.com/scalaz/releases",
"Sonatype OSS Snapshots" at "http://oss.sonatype.org/content/repositories/snapshots/",
"Typesafe repository" at "http://repo.typesafe.com/typesafe/releases/"
)
libraryDependencies ++= Seq(
"com.typesafe.play" %% "play-json" % "2.2.2-RC1",
"org.scalaz.stream" %% "scalaz-stream" % "0.3.1",
"org.scalaz" %% "scalaz-concurrent" % "7.0.6",
"org.typelevel" %% "shapeless-scalaz" % "0.2-SNAPSHOT"
)
import scalaz._, Scalaz._, concurrent.Task, stream.io.linesR
import shapeless._, contrib.scalaz._
import play.api.libs.json._, play.api.libs.functional.syntax._
case class Result(deletions: Int, entities: Int, colors: Map[String, Int])
object Result {
val deletion = Result(1, 0, Map.empty)
def tweet(entities: Int, color: String) =
Result(0, entities, (color != "C0DEED") ?? Map(color -> 1))
}
object PlayJsonDemo {
val entitySum = (
(__ \ 'hashtags ).json.pick[JsArray].map(_.value.size) and
(__ \ 'symbols ).json.pick[JsArray].map(_.value.size) and
(__ \ 'urls ).json.pick[JsArray].map(_.value.size) and
(__ \ 'user_mentions).json.pick[JsArray].map(_.value.size)
)((a, b, c, d) => a + b + c + d)
val resultReads = (__ \ 'delete).json.pick.map(_ => Result.deletion) or (
(__ \ 'entities).read(entitySum) and
(__ \ 'user \ 'profile_background_color).read[String]
)(Result.tweet _)
def result: Task[Result] = linesR(
"../sample-all-2014-03-16.json"
).gatherMap(16)(
line => Task(Json.parse(line).as(resultReads))
).runFoldMap(identity)
}
@travisbrown
Copy link
Author

In response to a challenge by Christopher Brown. On just over a million tweets on my dual-core laptop, the Play implementation takes about 39 seconds, Json4s about 38, and Argonaut about 51.

@mandubian
Copy link

It's cool to see that with functional composers from Play-Json, we don't lose much time compared to more "imperative" code like json4s (which you could also write for play-json)...
I wonder how it behaves with JsZipper ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment