파이썬 언어로 개발되었으며, 몽고 로그 파일 분석하여 Query 비용 분석과 시각화 도구를 제공한다.

# install

https://github.com/rueckstiess/mtools/releases

pip install mtools --upgrade

mloginfo mongod.log --queries

QUERIES

namespace                    pattern                                        count    min (ms)    max (ms)    mean (ms)    95%-ile (ms)    sum (ms)

serverside.scrum_master      {"datetime_used": {"$ne": 1}}                     20       15753       17083        16434         17039.3      328692

serverside.django_session    {"_id": 1}                                       562         101        1512          317           842.6      178168

serverside.user              {"_types": 1, "emails.email": 1}                 804         101        1262          201          684.85      162311

local.slaves                 {"_id": 1, "host": 1, "ns": 1}                   131         101        1048          310           819.5       40738

serverside.email_alerts      {"_types": 1, "email": 1, "pp_user_id": 1}        13         153       11639         2465          8865.2       32053

# 참고링크

http://blog.mongodb.org/post/85123256973/introducing-mtools 

CouchBase 4.0 개발자 버전이 릴리즈 되었다.

SQL 구문으로 CouchBase에 저장된 도큐먼트들을 조회할 수 있도록 기능이 개선되었다.

• http://blog.couchbase.com/introducing-developer-preview-for-couchbase-server-4.0
• http://docs.couchbase.com/4.0/getting-started/installing.html
• http://distrowatch.com/table.php?distribution=centos
• http://www.couchbase.com/nosql-databases/downloads#PreRelease

# OS CentOS7 지원
cat /etc/redhat-release
CentOS Linux release 7.1.1503 (Core)

rpm -qa *-release
centos-release-7-1.1503.el7.centos.2.8.x86_64

######################################################
# 설치시 주의해야 할 것..
# http://blog.couchbase.com/often-overlooked-linux-os-tweaks
######################################################

yum install wget 
 
mkdir -p /usr1/program
cd  /usr1/program
 
wget --continue http://packages.couchbase.com/releases/4.0.0-dp/couchbase-server-enterprise-4.0.0-dp-centos7.x86_64.rpm
sudo rpm --install couchbase-server-enterprise-4.0.0-dp-centos7.x86_64.rpm
 
service couchbase-server start 
 
 ######################################################
# PUBLIC ZONE 방화벽 오픈
# http://www.conory.com/note_linux/42477
# https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Security_Guide/sec-Using_Firewalls.html
######################################################
firewall-cmd --permanent --zone=public --add-port=8091/tcp
firewall-cmd --permanent --zone=public --add-port=8092/tcp
firewall-cmd --permanent --zone=public --add-port=11211/tcp
firewall-cmd --permanent --zone=public --add-port=11210/tcp
firewall-cmd --permanent --zone=public --add-port=11209/tcp
firewall-cmd --permanent --zone=public --add-port=4369/tcp
firewall-cmd --permanent --zone=public --add-port=21100-21299/tcp

firewall-cmd --reload

-- 방화벽 존의 설정 파일 확인 
/etc/firewalld/zones/public.xml

systemctl enable couchbase-server

######################################################
## new feature N1QL
## http://docs.couchbase.com/4.0/n1ql/getting-started.html
######################################################

cd /opt/couchbase/bin
./cbq -engine=http://localhost:8093

-- Settings > Sample Buckets 
beer-sample 버켓 생성

-- 1. primary key 생성
CREATE PRIMARY INDEX ON `beer-sample` USING GSI;

-- 2. grouping 
SELECT DISTINCT type FROM `beer-sample`;

-- 3. query 
SELECT * FROM `beer-sample` WHERE type="brewery" LIMIT 1;

오랜만에 몽고DB 3.0.0-rc8 버젼업되어서 설치해봅니다^^

/*
    mongodb 3.0.0 rc8 install

*/

# http://www.mongodb.org/dl/linux/x86_64
# 다운로드 
cd ~/
wget http://downloads.mongodb.org/linux/mongodb-linux-x86_64-3.0.0-rc8.tgz
tar -xf mongodb-linux-x86_64-3.0.0-rc8.tgz
mkdir -p /var/mongodb_3.0.0
mv mongodb-linux-x86_64-3.0.0-rc8/* /var/mongodb_3.0.0/


# 데이타 및 로그 폴더 설정 
mkdir -p /usr1/mongodb_3.0.0
mkdir -p /usr1/mongodb_3.0.0/logs

# 설정 파일 
vi /etc/mongodb_3.0.0.conf
---------------------------------------------------------------
fork = true
#bind_ip = 0.0.0.0
port = 9999
quiet = true
dbpath = /usr1/mongodb_3.0.0
logpath = /usr1/mongodb_3.0.0/logs/mongod.log
logappend = true
journal = true
#auth = true

# 실행 쉘 스크립트 변경 
---------------------------------------------------------------
wget https://ewan.im/wp-content/uploads/2010/04/mongodb.txt 
cp mongodb.txt /etc/init.d/mongodb3
chmod 755 /etc/init.d/mongodb3

>> 설정 파일 변경
mongod path 변경 및 conf 변경
---------------------------------------------------------------
vi /etc/init.d/mongodb3
    --config /etc/mongodb_3.0.0.conf

>> 서비스 시작 설정    
chkconfig --add mongodb3
chkconfig mongodb3 on   
  
# Starts up MongoDB right now
/etc/init.d/mongodb3 start
   
# 로그 확인 
tail -30 /usr1/mongodb_3.0.0/logs/mongod.log
---------------------------------------------------------------
2015-02-13T08:57:42.067+0900 I JOURNAL  [initandlisten] journal dir=/usr1/mongodb_3.0.0/journal
2015-02-13T08:57:42.067+0900 I JOURNAL  [initandlisten] recover : no journal files present, no recovery needed
2015-02-13T08:57:42.129+0900 I JOURNAL  [durability] Durability thread started
2015-02-13T08:57:42.129+0900 I JOURNAL  [journal writer] Journal writer thread started
2015-02-13T08:57:42.131+0900 I INDEX    [initandlisten] allocating new ns file /usr1/mongodb_3.0.0/local.ns, filling with zeroes...
2015-02-13T08:57:42.286+0900 I STORAGE  [FileAllocator] allocating new datafile /usr1/mongodb_3.0.0/local.0, filling with zeroes...
2015-02-13T08:57:42.286+0900 I STORAGE  [FileAllocator] creating directory /usr1/mongodb_3.0.0/_tmp
2015-02-13T08:57:42.352+0900 I STORAGE  [FileAllocator] done allocating datafile /usr1/mongodb_3.0.0/local.0, size: 64MB,  took 0.057 secs
2015-02-13T08:57:42.369+0900 I NETWORK  [initandlisten] waiting for connections on port 9999

# storage engine 변경해보자~~
vi /etc/init.d/mongodb3
--storageEngine wiredTiger
---------------------------------------------------------------
 # Source function library
 12 . /etc/rc.d/init.d/functions
 13
 14 if [ -f /etc/sysconfig/mongodb ]; then
 15     . /etc/sysconfig/mongodb
 16 fi
 17
 18 prog="mongod"
 19 mongod="/var/mongodb_3.0.0/bin/mongod"
 20 RETVAL=0
 21
 22 start() {
 23     echo -n $"Starting $prog: "
 24     daemon $mongod "--fork --storageEngine wiredTiger --config /etc/mongodb_3.0.0.conf"
 25     RETVAL=$?
 26     echo
 27     [ $RETVAL -eq 0 ] && touch /var/lock/subsys/$prog
 28     return $RETVAL
 29 }
---------------------------------------------------------------

# 서비스 재 시작~~
/etc/init.d/mongodb3 start

# 오류 발생 
tail -30 /usr1/mongodb_3.0.0/logs/mongod.log
Detected data files in /usr1/mongodb_3.0.0 created by storage engine 'mmapv1'. The configured storage engine is 'wiredTiger'., terminating

# 아하~~ 좋아~
# 1. 기존 폴더 삭제 
rm -rf  /usr1/mongodb_3.0.0

# 2. 폴더 재생성 
mkdir -p /usr1/mongodb_3.0.0
mkdir -p /usr1/mongodb_3.0.0/logs

# 서비스 재 시작~~
/etc/init.d/mongodb3 start
--------------

2015-02-13T09:05:13.514+0900 I CONTROL  [initandlisten] MongoDB starting : pid=6350 port=9999 dbpath=/usr1/mongodb_3.0.0 64-bit host=localhost.localdomain
2015-02-13T09:05:13.514+0900 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2015-02-13T09:05:13.514+0900 I CONTROL  [initandlisten]
2015-02-13T09:05:13.514+0900 I CONTROL  [initandlisten]
2015-02-13T09:05:13.514+0900 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2015-02-13T09:05:13.514+0900 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2015-02-13T09:05:13.514+0900 I CONTROL  [initandlisten]
2015-02-13T09:05:13.514+0900 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2015-02-13T09:05:13.514+0900 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2015-02-13T09:05:13.514+0900 I CONTROL  [initandlisten]
2015-02-13T09:05:13.514+0900 I CONTROL  [initandlisten] db version v3.0.0-rc8
2015-02-13T09:05:13.514+0900 I CONTROL  [initandlisten] git version: 9d0714cdcffadb203ff68730acedbc40875403da
2015-02-13T09:05:13.514+0900 I CONTROL  [initandlisten] build info: Linux build4.nj1.10gen.cc 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49
2015-02-13T09:05:13.515+0900 I CONTROL  [initandlisten] allocator: tcmalloc
2015-02-13T09:05:13.515+0900 I CONTROL  [initandlisten] options: { config: "/etc/mongodb_3.0.0.conf", net: { port: 9999 }, processManagement: { fork: true }, storage: { dbPath: "/usr1/mongodb_3.0.0", engine: "wiredTiger", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/usr1/mongodb_3.0.0/logs/mongod.log", quiet: true } }
2015-02-13T09:05:13.536+0900 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2015-02-13T09:05:13.730+0900 I NETWORK  [initandlisten] waiting for connections on port 9999

mongodb 2.8.0-rc4 몽고 성능 테스트 측정을 위해 sysbench-mongo라는 툴을 사용했다. 

https://github.com/tmcallaghan/sysbench-mongodb


1. 테스트 전 필요 설치 목록 

가. java sdk 설치 
    java sdk 1.6 or 1.7+ 이상 설치되어야 한다. 

    # 설치 가능한 목록 확인
    yum list java-*-devel

    # java sdk 설치 
    yum -y install java-1.6.0-openjdk-devel.x86_64

    [root@localhost ~]# java -version
    java version "1.6.0_33"
    OpenJDK Runtime Environment (IcedTea6 1.13.5) (rhel-1.13.5.1.el6_6-x86_64)
    OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)
    
나. mongo java driver 다운로드 설치 

    wget http://central.maven.org/maven2/org/mongodb/mongo-java-driver/2.11.4/mongo-java-driver-2.11.4.jar
    wget http://central.maven.org/maven2/org/mongodb/mongo-java-driver/2.13.0-rc1/mongo-java-driver-2.13.0-rc1.jar
    
다.  Sysbench 설치    

    git clone https://github.com/tmcallaghan/sysbench-mongodb.git

    # 2.4 수행시 설정 
    export CLASSPATH=$PWD/mongo-java-driver-2.11.4.jar:$CLASSPATH
    
    # 2.8 수행시 설정 
    export CLASSPATH=$PWD/mongo-java-driver-2.13.0-rc1.jar:$CLASSPATH

    #. 2.4와 2.8 함수명 변경으로 각각 수행시 드라이버 지정 변경해줘야 한다. 
    cp -r sysbench-mongodb  sysbench-mongodb-2.8    
    cd sysbench-mongodb        
    
    # 환경변수 해제 
    # unset CLASSPATH
    
    # java driver의 함수가 변경되었군!!
    https://github.com/mongodb/mongo-java-driver/releases

    Replace any calls to MongoCredential#createMongoCRCredential with calls to MongoCredential#createCredential.
    createMongoCRCredential -> createCredential     (함수명 변경)    
    
라. mongodb 2.8 rc4 가동 

    # 향상된 기능을 사용하기 위해서는  --storageEngine wiredtiger 엔진으로 옵션 지정해야 한다.    
    /var/mongodb_2.8.0/bin/mongod --config /etc/mongodb_2.8.0.conf --storageEngine wiredtiger
    
2. 테스트 진행

    # 환경 정보 설정 변경   (ip, port, collection 갯수 등등.. 설정 변경)
    vi config.bash

    테스트 실행
    ------------------------------------------------------------------------------------------

    # 2.8 버젼 
    ./run.simple.bash   config_28.bash

    # 2.4 버젼 
    ./run.simple.bash   config_24.bash    

    

원본 링크
http://cafe.naver.com/couchbase/1313

Couchbase 서버 3.0의 새로운 기능 – http://docs.couchbase.com/admin/admin/Whats-new-3.0.html

릴리스 3.0 노트 – http://docs.couchbase.com/admin/admin/rel-notes/rel-notes3.0.html

Couchbase 서버 3.0 다운로드 – http://www.couchbase.com/download

Couchbase 서버 3.0 메뉴얼 – http://docs.couchbase.com/admin

sudo yum install gcc gcc-c++ kernel-devel make ncurses-devel \
    openssl-devel rpm-build python-setuptools python-devel \
    git cyrus-sasl-devel openssl-devel libtool scons python-26

yum provides */libcrypto.so.6
sudo yum install openssl098e    

wget --continue http://packages.couchbase.com/releases/3.0.0/couchbase-server-community-3.0.0-centos6.x86_64.rpm
sudo rpm --install couchbase-server-community-3.0.0-centos6.x86_64.rpm

# 방화벽 오픈
iptables -L
iptables -I INPUT -p tcp --dport 8091 -j ACCEPT
iptables -I INPUT -p tcp --dport 8092 -j ACCEPT
iptables -I INPUT -p tcp --dport 11211 -j ACCEPT
iptables -I INPUT -p tcp --dport 11210 -j ACCEPT
iptables -I INPUT -p tcp --dport 11209 -j ACCEPT
iptables -I INPUT -p tcp --dport 4369 -j ACCEPT
iptables -I INPUT -p tcp --dport 21100:21299 -j ACCEPT

service iptables save
/etc/init.d/iptables restart

http://www.couchbase.com/presentations/couchbase-tlv-2014-couchbase-at-viber

viber라는 회사도 몽고DB에서 CouchBase로 마이그레이션했네,

본 문서는 여기 참고..

Couchbase vs. Apache CouchDB

간단하게 확장이 가능하네...hot spot이 전혀 발생을 안 하는군..  또한 설정 방법에 따라 메모리 저장소로 쓸수도 있고(memcached), 도큐먼트 형식인 CouchDB로 사용할 수도 있다.

카우치DB 한글 설명문
http://www.couchbase.com/kr/couchbase-server/why-couchbase

http://djcodeplus.wordpress.com/2014/01/08/couchbase-server-%EC%86%8C%EA%B0%9C-%EB%B0%8F-%EC%95%84%ED%82%A4%ED%85%8D%EC%B3%90-part1-13/
http://djcodeplus.wordpress.com/2014/01/08/couchbase-server-%EC%86%8C%EA%B0%9C-%EB%B0%8F-%EC%95%84%ED%82%A4%ED%85%8D%EC%B3%90-part1-23/

• CouchDB 개념 소개
  http://crazia.tistory.com/764

• 설치 및 다운로드
  http://docs.couchdb.org/en/latest/install/windows.html#installation-from-binaries

증상 : 잘 되던 몽고DB의 맵리듀스가 갑자기 실행이 안되는 문제가 발생햇다. mapreduce의 반환 오류는 property를 읽을 수 없다고 하는데 도대체 먼 문제인지 알 수가 없었고, 몽고DB 로그 읽어 확인을 했다.

Fri Dec 20 14:46:05.328 [conn712] JavaScript execution failed: TypeError: Cannot read property 'Computer' of undefined near 'log.PcInfo.Computer;         if (reval.ip'  (line 12)
Fri Dec 20 14:46:05.328 [conn712] warning: ClientCursor::YieldLock not closed properly

Fri Dec 20 14:46:05.328 [conn712] mr failed, removing collection :: caused by :: 16722 JavaScript execution failed: TypeError: Cannot read property 'Computer' of undefined near 'log.PcInfo.Computer;         if (reval.ip'  (line 12)
Fri Dec 20 14:46:05.801 [conn712] JavaScript execution failed: TypeError: Cannot read property 'Computer' of undefined near 'log.PcInfo.Computer;         if (reval.ip'  (line 12)
Fri Dec 20 14:46:05.801 [conn712] warning: ClientCursor::YieldLock not closed properly

해결 : Mapreduce의 value 값이 16M가 넘어서 그렇다고하는데 ㅡ.ㅡ 정확한 것인가? 일단 그래서 map의 value값을 this에서  필요한 값들로 변경해서 해결했다.

https://jira.mongodb.org/browse/SERVER-5452

this error means that one document being created during MR is larger than the max size of 16MB.
You should try to break any large document up in smaller ones with the emit key, or change the process in some way.

https://www.simple-talk.com/cloud/cloud-data/data-science-laboratory-system-%E2%80%93-graph-databases/