gitリポジトリのサイズを減らしたときのメモ

結果

↓こんな感じでやったらうまくいった。

git filter-branch --index-filter 'git rm -r --cached --ignore-unmatch \
aaa/bbb/ccc.swf \
aaa/bbb/ddd.swf \
aaa/bbb/eee.swf \
cdn_server/' \
--prune-empty --tag-name-filter cat -- --all

git gc --prune=now
git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d
git reflog expire --expire=now --all
git gc --prune=now

push

サイズが減った後のoriginへのpush

git push origin --force --all
git push origin --force --tags

http://stevelorek.com/how-to-shrink-a-git-repository.html

参考にしたURL

https://confluence.atlassian.com/bitbucket/reduce-repository-size-321848262.html

http://easyramble.com/git-filter-branch.html

https://git-scm.com/book/ja/v1/Git%E3%81%AE%E5%86%85%E5%81%B4-%E3%83%A1%E3%82%A4%E3%83%B3%E3%83%86%E3%83%8A%E3%83%B3%E3%82%B9%E3%81%A8%E3%83%87%E3%83%BC%E3%82%BF%E3%83%AA%E3%82%AB%E3%83%90%E3%83%AA

http://stackoverflow.com/questions/10656794/why-do-large-files-still-exist-in-my-packfile-after-scrubbing-them-with-filter

http://dskd.jp/archives/46.html

ハマりポイント4つ メモ

  • ファイルパスが間違っていると、当然、消えない。

git filter-branchを実行するとヒットしなくても処理が行われてる風な出力がある。でもサイズは減らない。

  • git filter-branch をしたら最初に git gc –prune=now を実行する。

atlassianの記事やstackoverflowの回答だと、
最後にgit gc –prune=nowするだけで良さそうだけど、
試しに、小さいgitリポジトリを作って試したら、
git gc –prune=nowをやらないと
.git/objects/ の中が空だった。
空のままで、

git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d
git reflog expire --expire=now --all

をやっても効果がないっぽい。

  • 最後に – –all を付ける。 git filter-branch –index-filter ‘git rm –cached –ignore-unmatch test1.swf’ –prune-empty – –all のように、最後に
    – –all を付ける。
    これをやらないと、別のブランチでログが残ったままになってリポジトリサイズが減らない。

  • –tag-name-filter cat を付ける。

git filter-branch –index-filter ‘git rm –cached –ignore-unmatch \ test1.swf \ test2.swf \ test3.swf’ \ –prune-empty –tag-name-filter cat – –all のように、
–tag-name-filter cat
を付ける。 これをやらないとタグのログが残ってリポジトリサイズが減らない。

Ref 'refs/tags/1.7.12' was rewritten
WARNING: You said to rewrite tagged commits, but not the corresponding tag.
WARNING: Perhaps use '--tag-name-filter cat' to rewrite the tag.

サイズが減らないときの確認方法

git log aabbcc.swf  

で、gitログを見て、履歴があったら、そもそもログから消えていない。git filter-branchがうまくいっていない。

git_find_big.sh
で表示されていたら .git/objects/ の中にデータが残っている。

du -hs .
du  .git/objects/

履歴を消したファイルをコピーして元に戻す

git rm --cached  test1.txt

のようにするとファイルは消えないけど、
git filter-branch の中で–cached を付けてもファイルが消える。

git filter-branch --index-filter 'git rm --cached --ignore-unmatch test1.txt' --prune-empty --tag-name-filter cat -- --all

今回はバイナリファイルの履歴を消したいだけだったので、
消えたファイルは、バックアップしていた同じリポジトリから、cpコマンドでコピーする。

Ansible --start-at-task

Ansible には –start-at-task という、
指定したタスクから開始するオプションがあるけどうまく動かない。

https://github.com/ansible/ansible/issues/11692

deploy% touch ansible.cfg
deploy% vi ansible.cfg

[defaults]
task_includes_static = True
handler_includes_static = True
% ansible-playbook -i inventory/hosts/localvm playbook/deploy.yml --start-at-task="Display all variables/facts known for a host"

で途中から実行できた。

TASK [carlosbuenosvinos.ansistrano-deploy : Display all variables/facts known for a host] ***
ok: [192.168.3.43] => {
    "hostvars[inventory_hostname]": {
        "ansible_all_ipv4_addresses": [
            "192.168.3.43"
...

メモ

https://github.com/ansible/ansible/issues/4561

roleを指定しているとダメだというチケットがあったので、
| で区切って指定したけど動かなかった。

–start-at-task=“carlosbuenosvinos.ansistrano-deploy : Display all variables/facts known for a host”
–start-at-task=“carlosbuenosvinos.ansistrano-deploy | Display all variables/facts known for a host”

ansibleを2.1.0 から ansible-2.2.1.0 にしてもansible.cfgを書かないと–start-at-taskは動かなかった。

ansible をアップグレードしようとしたらエラー

pip install –upgrade ansible でエラー。

pip install --upgrade ansible
Collecting ansible
  Downloading ansible-2.2.1.0.tar.gz (2.5MB)
    100% |################################| 2.5MB 404kB/s 


  build/temp.macosx-10.10-x86_64-2.7/_openssl.c:1107:27: error: unknown type name 'EVP_PKEY_CTX'
  int EVP_PKEY_encrypt_init(EVP_PKEY_CTX *);
                            ^
  fatal error: too many errors emitted, stopping now [-ferror-limit=]
  20 errors generated.
  error: command 'clang' failed with exit status 1
  
  ----------------------------------------
  Failed building wheel for cryptography
  Running setup.py clean for cryptography



    creating build/temp.macosx-10.10-x86_64-2.7/build/temp.macosx-10.10-x86_64-2.7
    clang -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/kubotad/.pyenv/versions/2.7.9/include/python2.7 -c build/temp.macosx-10.10-x86_64-2.7/_openssl.c -o build/temp.macosx-10.10-x86_64-2.7/build/temp.macosx-10.10-x86_64-2.7/_openssl.o
    build/temp.macosx-10.10-x86_64-2.7/_openssl.c:680:30: error: expected function body after function declarator
    typedef LHASH_OF(CONF_VALUE) Cryptography_LHASH_OF_CONF_VALUE;
                                 ^
    build/temp.macosx-10.10-x86_64-2.7/_openssl.c:886:5: error: expected ')'
    DH *DHparams_dup(DH *);
        ^
    /usr/include/openssl/dh.h:164:25: note: expanded from macro 'DHparams_dup'
    #define DHparams_dup(x) ASN1_dup_of_const(DH,i2d_DHparams,d2i_DHparams,x)
                            ^


Command "/Users/kubotad/.pyenv/versions/app1/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip-build-UehdPg/cryptography/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip-bUFtd6-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/kubotad/.pyenv/versions/app1/include/site/python2.7/cryptography" failed with error code 1 in /private/var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip-build-UehdPg/cryptography/

↑このコマンドは、後で気づいたけど
/Users/kubotad/PycharmProjects/app1/.python-version
というファイルがある場所で実行した。
なので、pyenvの中のpythonバージョンで実行されている。

sudoでやってみても失敗。

sudo pip install --upgrade ansible
Password:
The directory '/Users/kubotad/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/kubotad/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.


                        ^
    build/temp.macosx-10.10-x86_64-2.7/_openssl.c:1107:27: error: unknown type name 'EVP_PKEY_CTX'
    int EVP_PKEY_encrypt_init(EVP_PKEY_CTX *);
                              ^
    fatal error: too many errors emitted, stopping now [-ferror-limit=]
    20 errors generated.
    error: command 'clang' failed with exit status 1
    
    ----------------------------------------
Command "/Users/kubotad/.pyenv/versions/app1/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/tmp/pip-build-E79wWq/cryptography/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-wAEgKs-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/kubotad/.pyenv/versions/app1/include/site/python2.7/cryptography" failed with error code 1 in /private/tmp/pip-build-E79wWq/cryptography/

↓このコマンドは多分ホームディレクトリで実行した。

% pip install --upgrade ansible     
Downloading/unpacking ansible
  Downloading ansible-2.2.1.0.tar.gz (2.5MB): 2.5MB downloaded
  Running setup.py (path:/private/var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip_build_kubotad/ansible/setup.py) egg_info for package ansible



    changing mode of /Users/kubotad/.pyenv/versions/2.7.9/bin/ansible-pull to 755
    changing mode of /Users/kubotad/.pyenv/versions/2.7.9/bin/ansible-vault to 755
  Running setup.py install for PyYAML
    checking if libyaml is compilable
    clang -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/kubotad/.pyenv/versions/2.7.9/include/python2.7 -c build/temp.macosx-10.10-x86_64-2.7/check_libyaml.c -o build/temp.macosx-10.10-x86_64-2.7/check_libyaml.o
    build/temp.macosx-10.10-x86_64-2.7/check_libyaml.c:2:10: fatal error: 'yaml.h' file not found
    #include <yaml.h>
             ^
    1 error generated.
    
    libyaml is not found or a compiler error: forcing --without-libyaml
    (if libyaml is installed correctly, you may need to
     specify the option --include-dirs or uncomment and
     modify the parameter include_dirs in setup.cfg)
    
  Found existing installation: setuptools 7.0
    Uninstalling setuptools:
      Successfully uninstalled setuptools
  Running setup.py install for pycrypto
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/setuptools/__init__.py", line 10, in <module>
        from six.moves import filter, map
    ImportError: No module named six.moves
    Complete output from command /Users/kubotad/.pyenv/versions/2.7.9/bin/python2.7 -c "import setuptools, tokenize;__file__='/private/var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip_build_kubotad/pycrypto/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip-ijhgF2-record/install-record.txt --single-version-externally-managed --compile:
    Traceback (most recent call last):

  File "<string>", line 1, in <module>

  File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/setuptools/__init__.py", line 10, in <module>

    from six.moves import filter, map

ImportError: No module named six.moves

----------------------------------------
Cleaning up...
Command /Users/kubotad/.pyenv/versions/2.7.9/bin/python2.7 -c "import setuptools, tokenize;__file__='/private/var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip_build_kubotad/pycrypto/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip-ijhgF2-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /private/var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip_build_kubotad/pycrypto
Storing debug log for failure in /Users/kubotad/.pip/pip.log
~% pip install six
Downloading/unpacking six
  Downloading six-1.10.0-py2.py3-none-any.whl
Installing collected packages: six
Successfully installed six
Cleaning up...
~% pip install setuptools
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.pyenv/versions/2.7.9/lib/python2.7/site-packages
Cleaning up...
[kubotad@Kubotads-iMac](19:34:07)~% 

Ansibleの修復

関連パッケージを変えたので試しに、ansible のバージョンを確認したらエラー。壊れた。

~% ansible --version
ERROR! Unexpected Exception: No module named markupsafe
the full traceback was:

Traceback (most recent call last):
  File "/Users/kubotad/.pyenv/versions/2.7.9/bin/ansible", line 80, in <module>
    from ansible.cli.adhoc import AdHocCLI as mycli
  File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/ansible/cli/adhoc.py", line 28, in <module>
    from ansible.executor.task_queue_manager import TaskQueueManager
  File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py", line 29, in <module>
    from ansible.executor.play_iterator import PlayIterator
  File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/ansible/executor/play_iterator.py", line 30, in <module>
    from ansible.playbook.block import Block
  File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/ansible/playbook/__init__.py", line 27, in <module>
    from ansible.playbook.play import Play
  File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/ansible/playbook/play.py", line 27, in <module>
    from ansible.playbook.base import Base
  File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/ansible/playbook/base.py", line 29, in <module>
    from jinja2.exceptions import UndefinedError
  File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/jinja2/__init__.py", line 33, in <module>
    from jinja2.environment import Environment, Template
  File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/jinja2/environment.py", line 13, in <module>
    from jinja2 import nodes
  File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/jinja2/nodes.py", line 19, in <module>
    from jinja2.utils import Markup
  File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/jinja2/utils.py", line 531, in <module>
    from markupsafe import Markup, escape, soft_unicode
ImportError: No module named markupsafe
[kubotad@Kubotads-iMac](19:34:36)~% 
% pip install markupsafe
Downloading/unpacking markupsafe
  Downloading MarkupSafe-0.23.tar.gz
Cleaning up...
setuptools must be installed to install from a source distribution
Storing debug log for failure in /Users/kubotad/.pip/pip.log
% pip install -U pip
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
  Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB): 1.3MB downloaded
Installing collected packages: pip
  Found existing installation: pip 1.5.6
    Uninstalling pip:
      Successfully uninstalled pip
Successfully installed pip
Cleaning up...
[kubotad@Kubotads-iMac](19:39:23)~% 
[kubotad@Kubotads-iMac](19:39:24)~% 
[kubotad@Kubotads-iMac](19:39:24)~% pip install setuptools
Requirement already satisfied: setuptools in ./.pyenv/versions/2.7.9/lib/python2.7/site-packages
Requirement already satisfied: six>=1.6.0 in ./.pyenv/versions/2.7.9/lib/python2.7/site-packages (from setuptools)
Collecting appdirs>=1.4.0 (from setuptools)
  Using cached appdirs-1.4.0-py2.py3-none-any.whl
Collecting packaging>=16.8 (from setuptools)
  Using cached packaging-16.8-py2.py3-none-any.whl
Collecting pyparsing (from packaging>=16.8->setuptools)
  Using cached pyparsing-2.1.10-py2.py3-none-any.whl
Installing collected packages: appdirs, pyparsing, packaging
Successfully installed appdirs-1.4.0 packaging-16.8 pyparsing-2.1.10

markupsafe がインストールできた。

~% pip install markupsafe
Collecting markupsafe
Installing collected packages: markupsafe
Successfully installed markupsafe-0.23

直った。

% ansible --version     
ansible 2.2.1.0
  config file = 
  configured module search path = Default w/o overrides

app1 env

pyenv に入っているpythonでは、まだAnsibleが2.1.0.0のままだった。

pip install --upgrade cryptography

を実行するとエラーになる。

% pip install -U pip
Requirement already up-to-date: pip in /Users/kubotad/.pyenv/versions/app1/lib/python2.7/site-packages

ansible==2.2.1.0 の環境

% pip freeze
ansible==2.2.1.0
appdirs==1.4.0
Jinja2==2.8.1
MarkupSafe==0.23
packaging==16.8
paramiko==2.1.1
pyparsing==2.1.10
PyYAML==3.12
six==1.10.0
virtualenv==15.0.2

ansible==2.1.0.0 の環境

% pip freeze        
amqp==1.4.9
ansible==2.1.0.0
anyjson==0.3.3
appdirs==1.4.0
billiard==3.3.0.23
boto==2.41.0
bpmappers==0.8.2
celery==3.1.23
cffi==1.9.1
Django==1.8.13
django-celery==3.1.17
django-kombu==0.9.4
ecdsa==0.13
enum34==1.1.6
Fabric==1.11.1
idna==2.2
ipaddress==1.0.18
Jinja2==2.8.1
kombu==3.0.35
MarkupSafe==0.23
packaging==16.8
paramiko==1.17.1
psycopg2==2.6.2
pyasn1==0.2.2
pycparser==2.17
pycrypto==2.6.1
pyparsing==2.1.10
pytz==2016.4
PyYAML==3.12
redis==2.10.5
six==1.10.0

openssl

openssl version

でバージョンを確認したら、0.9.8 だった。
upgradeしようとしてもすでにインストール済みというメッセージがでる。

http://stackoverflow.com/questions/36372000/warning-openssl-1-0-2g-already-installed-when-using-homebrew

http://stackoverflow.com/questions/15185661/update-openssl-on-os-x-with-homebrew

% brew link openssl --force
Linking /usr/local/Cellar/openssl/1.0.2h_1... 1601 symlinks created

opensslをアップグレードしたら、cryptography がインストールできるようになった。

pip install --upgrade cryptography

Successfully built cryptography
Installing collected packages: cryptography
Successfully installed cryptography-1.7.2

ansible

% pip install --upgrade ansible
Collecting ansible


Installing collected packages: paramiko, ansible
  Found existing installation: paramiko 1.17.1
    Uninstalling paramiko-1.17.1:
      Successfully uninstalled paramiko-1.17.1
  Found existing installation: ansible 2.1.0.0
    Uninstalling ansible-2.1.0.0:
      Successfully uninstalled ansible-2.1.0.0
Successfully installed ansible-2.2.1.0 paramiko-2.1.1

うまくいった。

Amazon Linuxで残りディスク容量アラートを設定

メトリクス送信設定

CloudWatchのメトリクスをLinuxから送信して、ディスク容量が減ったらアラートメールを送信するようにする。

Amazon Linux

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mon-scripts.html

http://docs.aws.amazon.com/ja_jp/AWSEC2/latest/UserGuide/mon-scripts.html

https://aws.amazon.com/code/8720044071969977

設定対象のLinuxで作業

Metrics通知ツール本体の前に、使っているライブラリ?をインストール。

sudo yum install perl-Switch perl-DateTime perl-Sys-Syslog perl-LWP-Protocol-https

本体をダウンロード。

mkdir ~/cloudwatch/
cd ~/cloudwatch/
curl http://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip -O

解凍して、aws-scripts-mon/ を /usr/local/cloudwatch/ の下に移動。

sudo mkdir /usr/local/cloudwatch
sudo mv aws-scripts-mon/ /usr/local/cloudwatch/

IAMユーザーを作成して、アクセスキーを設定。

cd /usr/local/cloudwatch/aws-scripts-mon
cp awscreds.template awscreds.conf
vi awscreds.conf

IAM

IAMユーザーにアタッチしたポリシー。

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "cloudwatch:PutMetricData",
                "cloudwatch:GetMetricStatistics",
                "cloudwatch:ListMetrics",
                "ec2:DescribeTags"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

↓だとダメだった。

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudwatch:PutMetricData",
        "cloudwatch:GetMetricStatistics",
        "cloudwatch:ListMetrics",
        "ec2:DescribeTags"
    ],
      "Resource": [
        "arn:aws:logs:*:*:*",
        "arn:aws:cloudwatch:*:*:*"        
    ]
  }
 ]
}

cronに設定

cronに設定前にコマンドラインから試す。

/usr/local/cloudwatch/aws-scripts-mon/mon-put-instance-data.pl --disk-path=/ --disk-space-util --disk-space-used --disk-space-avail

crontab -e

*/5 * * * * /usr/local/cloudwatch/aws-scripts-mon/mon-put-instance-data.pl --disk-path=/ --disk-space-util --disk-space-used --disk-space-avail --from-cron

以上でMetrics送信の設定は完了。
動いているかの確認は、AWS Management Console の CloudWatchで

Metrics > Linux System > Filesystem, InstanceId, MountPath

残りディスク容量が減った時のアラートメールは、AWS Consoleでやる。

Alarms mail

AWS Management Console の CloudWatchに行く(インスタンスが動いているリージョン)

Alarms > Create Alarm
Search Metrics

DiskSpaceUtilization

で検索。
チェックボックスをチェックして、Name, Description, >= 80
で作成。
メール送信の機能は AWS SNS の機能を使っているらしい。

メトリクスではなくてCloudWatchにログを出力(メモ)

CloudWatchにログを出力することもできるらしい。
この手順はMetrics送信の設定の前にやったけど、これじゃなかった。

この記事の通りにやった。
http://docs.aws.amazon.com/ja_jp/AmazonCloudWatch/latest/logs/QuickStartEC2Instance.html

2017-02-06 02:06:45,273 - cwlogs.threads - ERROR - 12442 - Thread-23 - Exception caught in <EventBatchPublisher(Thread-23, started daemon 139840766506752)>
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/cwlogs/threads.py", line 58, in run
    self._run()
  File "/usr/lib/python2.7/dist-packages/cwlogs/push.py", line 1391, in _run
    self._publish_event_batch()
  File "/usr/lib/python2.7/dist-packages/cwlogs/push.py", line 1198, in _publish_event_batch
    self.sequence_token = self._put_log_events(self.event_batch)
  File "/usr/lib/python2.7/dist-packages/cwlogs/push.py", line 1236, in _put_log_events
    response = self.logs_service.put_log_events(**params)
  File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 159, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 483, in _make_api_call
    operation_model, request_dict)
  File "/usr/lib/python2.7/dist-packages/botocore/endpoint.py", line 117, in make_request
    return self._send_request(request_dict, operation_model)
  File "/usr/lib/python2.7/dist-packages/botocore/endpoint.py", line 142, in _send_request
    request = self.create_request(request_dict, operation_model)
  File "/usr/lib/python2.7/dist-packages/botocore/endpoint.py", line 126, in create_request
    operation_name=operation_model.name)
  File "/usr/lib/python2.7/dist-packages/botocore/hooks.py", line 227, in emit
    return self._emit(event_name, kwargs)
  File "/usr/lib/python2.7/dist-packages/botocore/hooks.py", line 210, in _emit
    response = handler(**kwargs)
  File "/usr/lib/python2.7/dist-packages/botocore/signers.py", line 90, in handler
    return self.sign(operation_name, request)
  File "/usr/lib/python2.7/dist-packages/botocore/signers.py", line 147, in sign
    auth.add_auth(request)
  File "/usr/lib/python2.7/dist-packages/botocore/auth.py", line 314, in add_auth
    raise NoCredentialsError
NoCredentialsError: Unable to locate credentials

sudo vi /var/log/awslogs.log
を書き換えてリスタート
sudo service awslogs restart

2017-02-06 02:10:23,622 - cwlogs.push.reader - INFO - 12842 - Thread-4 - Start reading file from 0.
2017-02-06 02:10:25,109 - cwlogs.push.publisher - WARNING - 12842 - Thread-2 - Caught exception: An error occurred (AccessDeniedException) when calling the PutLogEvents operation: User: arn:aws:iam::9915xxxxxxxxxx:user/mmr2host-awslogs is not authorized to perform: logs:PutLogEvents on resource: arn:aws:logs:us-east-1:9915xxxxxxxxxx:log-group:/var/log/messages:log-stream:i-5ca0f181
2017-02-06 02:10:25,110 - cwlogs.threads - ERROR - 12842 - Thread-2 - Exception caught in <EventBatchPublisher(Thread-2, started daemon 139899077621504)>
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/cwlogs/threads.py", line 58, in run
    self._run()
  File "/usr/lib/python2.7/dist-packages/cwlogs/push.py", line 1391, in _run
    self._publish_event_batch()
  File "/usr/lib/python2.7/dist-packages/cwlogs/push.py", line 1198, in _publish_event_batch
    self.sequence_token = self._put_log_events(self.event_batch)
  File "/usr/lib/python2.7/dist-packages/cwlogs/push.py", line 1236, in _put_log_events
    response = self.logs_service.put_log_events(**params)
  File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 159, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 494, in _make_api_call
    raise ClientError(parsed_response, operation_name)
ClientError: An error occurred (AccessDeniedException) when calling the PutLogEvents operation: User: arn:aws:iam::9915xxxxxxxxxx:user/mmr2host-awslogs is not authorized to perform: logs:PutLogEvents on resource: arn:aws:logs:us-east-1:9915xxxxxxxxxx:log-group:/var/log/messages:log-stream:i-5ca0f181
2017-02-06 02:21:13,763 - cwlogs.push.reader - INFO - 12842 - Thread-24 - Replay events end at 14798.
2017-02-06 02:21:13,763 - cwlogs.push.reader - INFO - 12842 - Thread-24 - Start reading file from 0. 
2017-02-06 02:21:14,890 - cwlogs.push.publisher - WARNING - 12842 - Thread-23 - Caught exception: An error occurred (ResourceNotFoundException) when calling the PutLogEvents operation: The specified log group does not exist. 
2017-02-06 02:21:14,890 - cwlogs.push.batch - INFO - 12842 - Thread-23 - Creating log group /var/log/messages.
2017-02-06 02:21:15,081 - cwlogs.push.batch - INFO - 12842 - Thread-23 - Creating log stream i-5ca0f181.
2017-02-06 02:21:15,316 - cwlogs.push.publisher - INFO - 12842 - Thread-23 - Log group: /var/log/messages, log stream: i-5ca0f181, queue size: 0, Publish batch: {'skipped_events_count': 0, 'first_event': {'timestamp': 1486264861000, 'start_position': 0L, 'end_position': 151L}, 'fallback_events_count': 0, 'last_event': {'timestamp': 1486345988000, 'start_position': 14706L, 'end_position': 14798L}, 'source_id': '9eb733a4a488d353c22b89549724bc1e', 'num_of_events': 159, 'batch_size_in_bytes': 18773}
2017-02-06 02:21:19,437 - cwlogs.push.publisher - INFO - 12842 - Thread-23 - Log group: /var/log/messages, log stream: i-5ca0f181, queue size: 0, Publish batch: {'skipped_events_count': 0, 'first_event': {'timestamp': 1486347485000, 'start_position': 14798L, 'end_position': 14900L}, 'fallback_events_count': 0, 'last_event': {'timestamp': 1486347485000, 'start_position': 14984L, 'end_position': 15076L}, 'source_id': '9eb733a4a488d353c22b89549724bc1e', 'num_of_events': 3, 'batch_size_in_bytes': 353}

ログの出力場所

正しく設定できているはずなのに、CloudWatchに出てこないと思ったら、
us-east-1 (US East (N. Virginia)) に出来ていた。
監視対象は us-west-2 (US West (Oregon))にあるのに。
yumで入れた設定ファイルの初期値で region = us-east-1 と書いてある。

[plugins]
cwlogs = cwlogs
[default]
region = us-east-1

PostgreSQLのリプリケーションをしてみるテスト

準備

postgresql94
AWS上にt1.smallでPostgresqlが動いているAmazon Linuxを2台構築。

Amazon Linux

$ cat /etc/os-release 
NAME="Amazon Linux AMI"
VERSION="2016.09"
ID="amzn"
ID_LIKE="rhel fedora"
VERSION_ID="2016.09"
PRETTY_NAME="Amazon Linux AMI 2016.09"
ANSI_COLOR="0;33"
CPE_NAME="cpe:/o:amazon:linux:2016.09:ga"
HOME_URL="http://aws.amazon.com/amazon-linux-ami/"

参考にしたウェブサイト

PostgreSQL レプリケーション構成を組んでみる - tom__bo’s Blog

PostgreSQL9.4でストリーミングレプリケーションする - マコーの日記

エラー1

-bash-4.2$ pg_basebackup -h 172.40.24.192 -p 5432 -D /var/lib/pgsql94/data --xlog --progress --verbose
pg_basebackup: could not connect to server: could not connect to server: Connection timed out
    Is the server running on host "172.40.24.192" and accepting
    TCP/IP connections on port 5432?

AWS の security group に自分自身のsgを許可(5432)

エラー2

-bash-4.2$ pg_basebackup -h 172.40.24.192 -p 5432 -D /var/lib/pgsql94/data --xlog --progress --verbose
pg_basebackup: could not connect to server: FATAL:  no pg_hba.conf entry for replication connection from host "172.40.135.122", user "postgres", SSL off

Postgresql doesn't accept replication connection - Database Administrators Stack Exchange

-bash-4.2$ diff pg_hba.conf pg_hba.conf.2017203 
85,86d84
< host    all             all             172.40.135.122/32        trust
< host    replication     postgres        172.40.135.122/32        trust

pg_basebackup が動いた。

-bash-4.2$ pg_basebackup -h 172.40.24.192 -p 5432 -D /var/lib/pgsql94/data --xlog --progress --verbose
transaction log start point: 0/E000028 on timeline 1
51646/51646 kB (100%), 1/1 tablespace                                         
transaction log end point: 0/E000128
pg_basebackup: base backup completed

メモ

Amazon Linux の場合、
recovery.conf.sample

/usr/share/pgsql94/recovery.conf.sample
にあった。

エラー3

INSERT文が止まる。
アプリも動かない(nginx 504 error: Gateway Timeout)

app1=> select application_name, client_addr, backend_start, state, sync_state from pg_stat_replication;
 application_name | client_addr | backend_start | state | sync_state 
------------------+-------------+---------------+-------+------------
 slave01          |             |               |       | 
(1 row)

参考にしたウェブサイトと違って client_addr や state などが空になっていた。
ただ、この状態はエラーの原因ではなかった。

http://dba.stackexchange.com/questions/50281/postgresql-replication-pg-stat-replication-is-showing-empty-columns

psql から INSERTして応答がなかったので、controll + c した時のメッセージ。

Cancel request sent
WARNING:  canceling wait for synchronous replication due to user request
DETAIL:  The transaction has already committed locally, but might not have been replicated to the standby.
INSERT 0 1

このエラーは、
master db の postgresql.conf に書く、synchronous_standby_names の値と
slave db の recovery.conf に書く、primary_conninfo の中の値が違っていると発生するっぽい。
レコード自体は作られていた。

postgresql.conf in master db.
synchronous_standby_names = 'slave01' # standby servers that provide sync rep

recovery.conf in slave db.
primary_conninfo = 'host=172.40.24.192 port=5432 application_name=slave01'

cronで作成日時から一定期間過ぎたファイルを削除

メモ:
cronで作成日時から一定期間過ぎたファイルを削除

MAILTO=""
# delete files older than 3 days
# 1 3,15 * * * find /var/www/app1/shared/docs/ -maxdepth 2 -type f -ctime +3 -delete

1 3,15 * * * find /var/www/app1/shared/docs/input/ -maxdepth 1 -type f -ctime +3 -delete
5 3,15 * * * find /var/www/app1/shared/docs/output/ -maxdepth 1 -type f -cmin +720 -delete

ansistranoのafter_symlinkでsudo(become)を実行する

ansible の YAML

ansible ansistrano の after_symlink hook で
/etc/init.d/
シンボリックリンクを作る。

- name: initd symlink
  become: true
  file:
    state: link
    src: /var/www/app1/current/deploy/initd/app1_uwsgi
    #dest: /home/user1/app1_uwsgi
    dest: /etc/init.d/app1_uwsgi
    mode: "u=rwx,g=rx,o=rx"

sudoをするにはbecomeを使うと書いてあったけど、
指定の方法が分からなかった。
“配列の中にハッシュをネスト” という方法で書かないといけないっぽい。

http://docs.ansible.com/ansible/become.html

http://www.task-notes.com/entry/20150922/1442890800

ansible-playbook -i inventory/hosts/localvm playbook/deploy.yml -K

ansible-playbook コマンドの最後に -K を付けると最初にsudo passwordを聞かれる。
Ansible, sudoパスワード要求を忘れただけでめんどくなる - Goldstine研究所

エラーメッセージ

その1

fatal: [192.168.3.43]: FAILED! => {"failed": true, "reason": "no action detected in task. This often indicates a misspelled module name, or incorrect module path.\n\nThe error appears to have been in '/Users/aaa/PycharmProjects/app1/deploy/playbook/ansistrano/after_symlink.yml': line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- tasks:\n  ^ here\n\n\nThe error appears to have been in '/Users/aaa/PycharmProjects/app1/deploy/playbook/ansistrano/after_symlink.yml': line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- tasks:\n  ^ here\n"}

その2

fatal: [192.168.3.43]: FAILED! => {"failed": true, "reason": "Syntax Error while loading YAML.\n\n\nThe error appears to have been in '/Users/aaa/PycharmProjects/app1/deploy/playbook/ansistrano/after_symlink.yml': line 4, column 18, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n- name: create django static files\n    django_manage:\n                 ^ here\n"}

完成品

PycharmProjects/app1/deploy/playbook/ansistrano/after_symlink.yml

- name: create django static files
  django_manage:
    command: collectstatic
    app_path: "{{ ansistrano_deploy_to }}/current/"
    virtualenv: "/home/user1/.pyenv/versions/app1/"


- name: init.d app1_uwsgi symlink
  become: true
  file:
    state: link
    src: /var/www/app1/current/deploy/initd/app1_uwsgi
    #dest: /home/user1/app1_uwsgi
    dest: /etc/init.d/app1_uwsgi
    mode: "u=rwx,g=rx,o=rx"

- name: init.d app1_celeryd symlink
  become: true
  file:
    state: link
    src: /var/www/app1/current/deploy/initd/app1_celeryd
    dest: /etc/init.d/app1_celeryd
    mode: "u=rwx,g=rx,o=rx"

- name: restart app1_uwsgi
  become: true
  service:
    name: app1_uwsgi
    state: restarted

- name: restart app1_celeryd
  become: true
  service:
    name: app1_celeryd
    state: restarted