Skip to content

Instantly share code, notes, and snippets.

View shun0102's full-sized avatar

Shusuke Mikami shun0102

14 followers · 3 following
View GitHub Profile
@shun0102
shun0102 / gist:926a802e96e914f4ea41
Created March 16, 2016 05:16
2016-03-16 12:52:36.622 +0900: Embulk v0.8.7
2016-03-16 12:52:42.721 +0900 [INFO] (0001:transaction): Loaded plugin embulk-input-mysql (0.6.4)
2016-03-16 12:52:42.837 +0900 [INFO] (0001:transaction): Loaded plugin embulk-output-jdbc (0.5.0)
2016-03-16 12:52:43.020 +0900 [INFO] (0001:transaction): Fetch size is 10000. Using server-side prepared statement.
2016-03-16 12:52:43.648 +0900 [INFO] (0001:transaction): SQL: SELECT
2016-03-16 12:52:43.936 +0900 [INFO] (0001:transaction): Using local thread executor with max_threads=2 / tasks=1
2016-03-16 12:52:44.041 +0900 [INFO] (0001:transaction): Connecting to jdbc:hive2://had01:10000;AuthMech=2;UID=hive options {}
2016-03-16 12:52:45.333 +0900 [INFO] (0001:transaction): Using insert mode
2016-03-16 12:52:45.881 +0900 [INFO] (0001:transaction): SQL: DROP TABLE IF EXISTS `table_d88a19cf0e40_bl_tmp000`
2016-03-16 12:52:46.045 +0900 [INFO] (0001:transaction): > 0.16 seconds (-1 rows)
@shun0102
shun0102 / gist:2584066a98ba4d47996f
Created March 16, 2016 05:12
2016-03-16 12:52:36.622 +0900: Embulk v0.8.7
2016-03-16 12:52:42.721 +0900 [INFO] (0001:transaction): Loaded plugin embulk-input-mysql (0.6.4)
2016-03-16 12:52:42.837 +0900 [INFO] (0001:transaction): Loaded plugin embulk-output-jdbc (0.5.0)
2016-03-16 12:52:43.020 +0900 [INFO] (0001:transaction): Fetch size is 10000. Using server-side prepared statement.
2016-03-16 12:52:43.648 +0900 [INFO] (0001:transaction): SQL: SELECT
2016-03-16 12:52:43.936 +0900 [INFO] (0001:transaction): Using local thread executor with max_threads=2 / tasks=1
2016-03-16 12:52:44.041 +0900 [INFO] (0001:transaction): Connecting to jdbc:hive2://had01:10000;AuthMech=2;UID=hive options {}
2016-03-16 12:52:45.333 +0900 [INFO] (0001:transaction): Using insert mode
2016-03-16 12:52:45.881 +0900 [INFO] (0001:transaction): SQL: DROP TABLE IF EXISTS `table_d88a19cf0e40_bl_tmp000`
2016-03-16 12:52:46.045 +0900 [INFO] (0001:transaction): > 0.16 seconds (-1 rows)
@shun0102
shun0102 / guess.yml
Last active August 29, 2015 14:18
embulk guess guess.yml -g jsonl -o guess_output.yml
@shun0102
shun0102 / gist:e70ad92df864607ea5cf
Last active August 29, 2015 14:16
var url = 'http://104.154.70.25/?callback=?';
$.getJSON(url, function(data){
console.log(data);
});
@shun0102
shun0102 / gist:bdf732df8470b3a73143
Created March 6, 2015 05:01
開始日,2015-01-01
終了日,2015-01-31
時間,URL,参照元,...
2015-01-01 00:00:00, http://example.com, ,...
2015-01-01 00:00:00, http://example.com, ,...
@shun0102
shun0102 / gist:3036fb86437d9974f872
Created February 16, 2015 05:09
embulk-work java -jar embulk.jar gem list
*** LOCAL GEMS ***
embulk-input-jdbc (0.1.0)
embulk-input-mysql (0.1.0)
embulk-output-mysql (0.1.0)
embulk-plugin-redis (0.1.2)
ffi (1.9.3 java)
jar-dependencies (0.1.2)
@shun0102
shun0102 / gist:a36109ae5e1626c8bba8
Created February 4, 2015 15:08
use timestamp at embulk input plugin 
def self.transaction(config, &control)
  columns = [Column.new(0, 'time', :timestamp)]
end

def run
  @page_builder.add([Time.now])
end
@shun0102
shun0102 / input_mysql.rb
Last active August 29, 2015 14:14
embulk mysql input plugin example
module Embulk
class InputMysql < InputPlugin
require 'jdbc/mysql'
Jdbc::MySQL.load_driver
Plugin.register_input('mysql', self)
def self.transaction(config, &control)
task = {
'host' => config.param('host', :string, :default => 'localhost'),
@shun0102
shun0102 / mongodbのエラー
Created August 23, 2012 10:31
Thu Aug 23 02:10:45 [conn262470] authenticate: { authenticate: 1, nonce: "e9b20d11d36d8841", user: "__system", key: "721ad512d6133393a33ffcdf72e8f613" }
Thu Aug 23 02:10:55 Invalid access at address: 0x7f54f4d751d8
Thu Aug 23 02:10:55 Got signal: 7 (Bus error).
Thu Aug 23 02:10:55 Backtrace:
0xa95ce9 0xa9640c 0x3e3f80f4a0 0x85f390 0x8a7e98 0x8b4439 0x8b5b3b 0x961b5d 0x963e25 0x83355f 0x822b1b 0x826805 0x827c85 0x827cd8 0x828170 0xaabca0 0x3e3f8077f1 0x3e3f4e5ccd
/usr/bin/mongod(_ZN5mongo10abruptQuitEi+0x3a9) [0xa95ce9]
/usr/bin/mongod(_ZN5mongo24abruptQuitWithAddrSignalEiP7siginfoPv+0x22c) [0xa9640c]
/lib64/libpthread.so.0() [0x3e3f80f4a0]
@shun0102
shun0102 / solrのDateFormatTransformerに関して
Created July 24, 2012 04:45
data-config.xmlの設定
<field column="ts" name="ts" dateTimeFormat="yyyy-MM-dd hh:mm:ss" locale="ja" />
以下の様な対応で日付が変換される
2012-03-02 00:00:00 => 2012-03-02T15:00:00Z
2012-03-03 12:00:00 => 2012-03-02T15:00:00Z
2012-03-03 00:00:00 => 2012-03-02T15:00:00Z