This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import datetime | |
def period_interval(date, period, n0, n1): | |
"""Return a 2-tuple of [start_date, end_date], | |
date: base date from which to calculate forwards/backwards | |
period: timedelta | |
n0: int, starting period number | |
n1: int, ending period number""" | |
return [date + (period * n0), |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The core of what the system is about is: | |
- Having rough/flexible data strutures | |
- Performing transformations on those data structures | |
- Finding the core "must be correct" structures we want to reason about | |
- Defining and naming those structures with relative precision, | |
hardening them into definite declarations | |
- Validating and verifying that input and output conform to those | |
now-hardened structures. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Thank you! These are great. | |
Here's my gut investigation so far: | |
The multi-item vs. single-item distinction is a really good point. In | |
things like GMail, it's a multi-item system, there's a breakdown between: | |
- the items you can select | |
- a small set of operations you can apply to the items in bulk |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rpm -Uvh http://archive.cloudera.com/cdh4/one-click-install/redhat/6/x86_64/cloudera-cdh-4-0.x86_64.rpm | |
yum -y install hadoop-0.20-conf-pseudo hbase hbase-regionserver hbase-master | |
sudo -u hdfs hdfs namenode -format | |
for x in `cd /etc/init.d ; ls hadoop-hdfs-*` ; do sudo service $x start ; done | |
sudo -u hdfs hadoop fs -mkdir /tmp | |
sudo -u hdfs hadoop fs -chmod -R 1777 /tmp | |
sudo -u hdfs hadoop fs -mkdir -p /var/lib/hadoop-hdfs/cache/mapred/mapred/staging | |
sudo -u hdfs hadoop fs -chmod 1777 /var/lib/hadoop-hdfs/cache/mapred/mapred/staging | |
sudo -u hdfs hadoop fs -chown -R mapred /var/lib/hadoop-hdfs/cache/mapred | |
for x in `cd /etc/init.d ; ls hadoop-0.20-mapreduce-*` ; do sudo service $x start ; done |