gitリポジトリのサイズを減らしたときのメモ
結果
↓こんな感じでやったらうまくいった。
git filter-branch --index-filter 'git rm -r --cached --ignore-unmatch \ aaa/bbb/ccc.swf \ aaa/bbb/ddd.swf \ aaa/bbb/eee.swf \ cdn_server/' \ --prune-empty --tag-name-filter cat -- --all git gc --prune=now git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d git reflog expire --expire=now --all git gc --prune=now
push
サイズが減った後のoriginへのpush
git push origin --force --all git push origin --force --tags
http://stevelorek.com/how-to-shrink-a-git-repository.html
参考にしたURL
https://confluence.atlassian.com/bitbucket/reduce-repository-size-321848262.html
http://easyramble.com/git-filter-branch.html
http://dskd.jp/archives/46.html
ハマりポイント4つ メモ
- ファイルパスが間違っていると、当然、消えない。
git filter-branchを実行するとヒットしなくても処理が行われてる風な出力がある。でもサイズは減らない。
- git filter-branch をしたら最初に git gc –prune=now を実行する。
atlassianの記事やstackoverflowの回答だと、
最後にgit gc –prune=nowするだけで良さそうだけど、
試しに、小さいgitリポジトリを作って試したら、
git gc –prune=nowをやらないと
.git/objects/ の中が空だった。
空のままで、
git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d git reflog expire --expire=now --all
をやっても効果がないっぽい。
最後に – –all を付ける。 git filter-branch –index-filter ‘git rm –cached –ignore-unmatch test1.swf’ –prune-empty – –all のように、最後に
– –all を付ける。
これをやらないと、別のブランチでログが残ったままになってリポジトリサイズが減らない。–tag-name-filter cat を付ける。
git filter-branch –index-filter ‘git rm –cached –ignore-unmatch \
test1.swf \
test2.swf \
test3.swf’ \
–prune-empty –tag-name-filter cat – –all
のように、
–tag-name-filter cat
を付ける。
これをやらないとタグのログが残ってリポジトリサイズが減らない。
Ref 'refs/tags/1.7.12' was rewritten WARNING: You said to rewrite tagged commits, but not the corresponding tag. WARNING: Perhaps use '--tag-name-filter cat' to rewrite the tag.
サイズが減らないときの確認方法
git log aabbcc.swf
で、gitログを見て、履歴があったら、そもそもログから消えていない。git filter-branchがうまくいっていない。
git_find_big.sh
で表示されていたら .git/objects/ の中にデータが残っている。
du -hs . du .git/objects/
履歴を消したファイルをコピーして元に戻す
git rm --cached test1.txt
のようにするとファイルは消えないけど、
git filter-branch の中で–cached を付けてもファイルが消える。
git filter-branch --index-filter 'git rm --cached --ignore-unmatch test1.txt' --prune-empty --tag-name-filter cat -- --all
今回はバイナリファイルの履歴を消したいだけだったので、
消えたファイルは、バックアップしていた同じリポジトリから、cpコマンドでコピーする。
Ansible --start-at-task
Ansible には –start-at-task という、
指定したタスクから開始するオプションがあるけどうまく動かない。
https://github.com/ansible/ansible/issues/11692
deploy% touch ansible.cfg
deploy% vi ansible.cfg
[defaults] task_includes_static = True handler_includes_static = True
% ansible-playbook -i inventory/hosts/localvm playbook/deploy.yml --start-at-task="Display all variables/facts known for a host"
で途中から実行できた。
TASK [carlosbuenosvinos.ansistrano-deploy : Display all variables/facts known for a host] *** ok: [192.168.3.43] => { "hostvars[inventory_hostname]": { "ansible_all_ipv4_addresses": [ "192.168.3.43" ...
メモ
https://github.com/ansible/ansible/issues/4561
roleを指定しているとダメだというチケットがあったので、
| で区切って指定したけど動かなかった。
–start-at-task=“carlosbuenosvinos.ansistrano-deploy : Display all variables/facts known for a host”
–start-at-task=“carlosbuenosvinos.ansistrano-deploy | Display all variables/facts known for a host”
ansibleを2.1.0 から ansible-2.2.1.0 にしてもansible.cfgを書かないと–start-at-taskは動かなかった。
ansible をアップグレードしようとしたらエラー
pip install –upgrade ansible でエラー。
pip install --upgrade ansible Collecting ansible Downloading ansible-2.2.1.0.tar.gz (2.5MB) 100% |################################| 2.5MB 404kB/s build/temp.macosx-10.10-x86_64-2.7/_openssl.c:1107:27: error: unknown type name 'EVP_PKEY_CTX' int EVP_PKEY_encrypt_init(EVP_PKEY_CTX *); ^ fatal error: too many errors emitted, stopping now [-ferror-limit=] 20 errors generated. error: command 'clang' failed with exit status 1 ---------------------------------------- Failed building wheel for cryptography Running setup.py clean for cryptography creating build/temp.macosx-10.10-x86_64-2.7/build/temp.macosx-10.10-x86_64-2.7 clang -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/kubotad/.pyenv/versions/2.7.9/include/python2.7 -c build/temp.macosx-10.10-x86_64-2.7/_openssl.c -o build/temp.macosx-10.10-x86_64-2.7/build/temp.macosx-10.10-x86_64-2.7/_openssl.o build/temp.macosx-10.10-x86_64-2.7/_openssl.c:680:30: error: expected function body after function declarator typedef LHASH_OF(CONF_VALUE) Cryptography_LHASH_OF_CONF_VALUE; ^ build/temp.macosx-10.10-x86_64-2.7/_openssl.c:886:5: error: expected ')' DH *DHparams_dup(DH *); ^ /usr/include/openssl/dh.h:164:25: note: expanded from macro 'DHparams_dup' #define DHparams_dup(x) ASN1_dup_of_const(DH,i2d_DHparams,d2i_DHparams,x) ^ Command "/Users/kubotad/.pyenv/versions/app1/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip-build-UehdPg/cryptography/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip-bUFtd6-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/kubotad/.pyenv/versions/app1/include/site/python2.7/cryptography" failed with error code 1 in /private/var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip-build-UehdPg/cryptography/
↑このコマンドは、後で気づいたけど
/Users/kubotad/PycharmProjects/app1/.python-version
というファイルがある場所で実行した。
なので、pyenvの中のpythonバージョンで実行されている。
sudoでやってみても失敗。
sudo pip install --upgrade ansible Password: The directory '/Users/kubotad/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. The directory '/Users/kubotad/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. ^ build/temp.macosx-10.10-x86_64-2.7/_openssl.c:1107:27: error: unknown type name 'EVP_PKEY_CTX' int EVP_PKEY_encrypt_init(EVP_PKEY_CTX *); ^ fatal error: too many errors emitted, stopping now [-ferror-limit=] 20 errors generated. error: command 'clang' failed with exit status 1 ---------------------------------------- Command "/Users/kubotad/.pyenv/versions/app1/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/tmp/pip-build-E79wWq/cryptography/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-wAEgKs-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/kubotad/.pyenv/versions/app1/include/site/python2.7/cryptography" failed with error code 1 in /private/tmp/pip-build-E79wWq/cryptography/
↓このコマンドは多分ホームディレクトリで実行した。
% pip install --upgrade ansible Downloading/unpacking ansible Downloading ansible-2.2.1.0.tar.gz (2.5MB): 2.5MB downloaded Running setup.py (path:/private/var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip_build_kubotad/ansible/setup.py) egg_info for package ansible changing mode of /Users/kubotad/.pyenv/versions/2.7.9/bin/ansible-pull to 755 changing mode of /Users/kubotad/.pyenv/versions/2.7.9/bin/ansible-vault to 755 Running setup.py install for PyYAML checking if libyaml is compilable clang -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/kubotad/.pyenv/versions/2.7.9/include/python2.7 -c build/temp.macosx-10.10-x86_64-2.7/check_libyaml.c -o build/temp.macosx-10.10-x86_64-2.7/check_libyaml.o build/temp.macosx-10.10-x86_64-2.7/check_libyaml.c:2:10: fatal error: 'yaml.h' file not found #include <yaml.h> ^ 1 error generated. libyaml is not found or a compiler error: forcing --without-libyaml (if libyaml is installed correctly, you may need to specify the option --include-dirs or uncomment and modify the parameter include_dirs in setup.cfg) Found existing installation: setuptools 7.0 Uninstalling setuptools: Successfully uninstalled setuptools Running setup.py install for pycrypto Traceback (most recent call last): File "<string>", line 1, in <module> File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/setuptools/__init__.py", line 10, in <module> from six.moves import filter, map ImportError: No module named six.moves Complete output from command /Users/kubotad/.pyenv/versions/2.7.9/bin/python2.7 -c "import setuptools, tokenize;__file__='/private/var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip_build_kubotad/pycrypto/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip-ijhgF2-record/install-record.txt --single-version-externally-managed --compile: Traceback (most recent call last): File "<string>", line 1, in <module> File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/setuptools/__init__.py", line 10, in <module> from six.moves import filter, map ImportError: No module named six.moves ---------------------------------------- Cleaning up... Command /Users/kubotad/.pyenv/versions/2.7.9/bin/python2.7 -c "import setuptools, tokenize;__file__='/private/var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip_build_kubotad/pycrypto/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip-ijhgF2-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /private/var/folders/1s/14vkt60n019bjctmy0rvg6cm0000gn/T/pip_build_kubotad/pycrypto Storing debug log for failure in /Users/kubotad/.pip/pip.log
~% pip install six Downloading/unpacking six Downloading six-1.10.0-py2.py3-none-any.whl Installing collected packages: six Successfully installed six Cleaning up...
~% pip install setuptools Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.pyenv/versions/2.7.9/lib/python2.7/site-packages Cleaning up... [kubotad@Kubotads-iMac](19:34:07)~%
Ansibleの修復
関連パッケージを変えたので試しに、ansible のバージョンを確認したらエラー。壊れた。
~% ansible --version ERROR! Unexpected Exception: No module named markupsafe the full traceback was: Traceback (most recent call last): File "/Users/kubotad/.pyenv/versions/2.7.9/bin/ansible", line 80, in <module> from ansible.cli.adhoc import AdHocCLI as mycli File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/ansible/cli/adhoc.py", line 28, in <module> from ansible.executor.task_queue_manager import TaskQueueManager File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py", line 29, in <module> from ansible.executor.play_iterator import PlayIterator File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/ansible/executor/play_iterator.py", line 30, in <module> from ansible.playbook.block import Block File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/ansible/playbook/__init__.py", line 27, in <module> from ansible.playbook.play import Play File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/ansible/playbook/play.py", line 27, in <module> from ansible.playbook.base import Base File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/ansible/playbook/base.py", line 29, in <module> from jinja2.exceptions import UndefinedError File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/jinja2/__init__.py", line 33, in <module> from jinja2.environment import Environment, Template File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/jinja2/environment.py", line 13, in <module> from jinja2 import nodes File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/jinja2/nodes.py", line 19, in <module> from jinja2.utils import Markup File "/Users/kubotad/.pyenv/versions/2.7.9/lib/python2.7/site-packages/jinja2/utils.py", line 531, in <module> from markupsafe import Markup, escape, soft_unicode ImportError: No module named markupsafe [kubotad@Kubotads-iMac](19:34:36)~%
% pip install markupsafe Downloading/unpacking markupsafe Downloading MarkupSafe-0.23.tar.gz Cleaning up... setuptools must be installed to install from a source distribution Storing debug log for failure in /Users/kubotad/.pip/pip.log
% pip install -U pip Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144 Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB): 1.3MB downloaded Installing collected packages: pip Found existing installation: pip 1.5.6 Uninstalling pip: Successfully uninstalled pip Successfully installed pip Cleaning up... [kubotad@Kubotads-iMac](19:39:23)~% [kubotad@Kubotads-iMac](19:39:24)~% [kubotad@Kubotads-iMac](19:39:24)~% pip install setuptools Requirement already satisfied: setuptools in ./.pyenv/versions/2.7.9/lib/python2.7/site-packages Requirement already satisfied: six>=1.6.0 in ./.pyenv/versions/2.7.9/lib/python2.7/site-packages (from setuptools) Collecting appdirs>=1.4.0 (from setuptools) Using cached appdirs-1.4.0-py2.py3-none-any.whl Collecting packaging>=16.8 (from setuptools) Using cached packaging-16.8-py2.py3-none-any.whl Collecting pyparsing (from packaging>=16.8->setuptools) Using cached pyparsing-2.1.10-py2.py3-none-any.whl Installing collected packages: appdirs, pyparsing, packaging Successfully installed appdirs-1.4.0 packaging-16.8 pyparsing-2.1.10
markupsafe がインストールできた。
~% pip install markupsafe Collecting markupsafe Installing collected packages: markupsafe Successfully installed markupsafe-0.23
直った。
% ansible --version ansible 2.2.1.0 config file = configured module search path = Default w/o overrides
app1 env
pyenv に入っているpythonでは、まだAnsibleが2.1.0.0のままだった。
pip install --upgrade cryptography
を実行するとエラーになる。
% pip install -U pip Requirement already up-to-date: pip in /Users/kubotad/.pyenv/versions/app1/lib/python2.7/site-packages
ansible==2.2.1.0 の環境
% pip freeze ansible==2.2.1.0 appdirs==1.4.0 Jinja2==2.8.1 MarkupSafe==0.23 packaging==16.8 paramiko==2.1.1 pyparsing==2.1.10 PyYAML==3.12 six==1.10.0 virtualenv==15.0.2
ansible==2.1.0.0 の環境
% pip freeze amqp==1.4.9 ansible==2.1.0.0 anyjson==0.3.3 appdirs==1.4.0 billiard==3.3.0.23 boto==2.41.0 bpmappers==0.8.2 celery==3.1.23 cffi==1.9.1 Django==1.8.13 django-celery==3.1.17 django-kombu==0.9.4 ecdsa==0.13 enum34==1.1.6 Fabric==1.11.1 idna==2.2 ipaddress==1.0.18 Jinja2==2.8.1 kombu==3.0.35 MarkupSafe==0.23 packaging==16.8 paramiko==1.17.1 psycopg2==2.6.2 pyasn1==0.2.2 pycparser==2.17 pycrypto==2.6.1 pyparsing==2.1.10 pytz==2016.4 PyYAML==3.12 redis==2.10.5 six==1.10.0
openssl
openssl version
でバージョンを確認したら、0.9.8 だった。
upgradeしようとしてもすでにインストール済みというメッセージがでる。
http://stackoverflow.com/questions/15185661/update-openssl-on-os-x-with-homebrew
% brew link openssl --force Linking /usr/local/Cellar/openssl/1.0.2h_1... 1601 symlinks created
opensslをアップグレードしたら、cryptography がインストールできるようになった。
pip install --upgrade cryptography Successfully built cryptography Installing collected packages: cryptography Successfully installed cryptography-1.7.2
ansible
% pip install --upgrade ansible Collecting ansible Installing collected packages: paramiko, ansible Found existing installation: paramiko 1.17.1 Uninstalling paramiko-1.17.1: Successfully uninstalled paramiko-1.17.1 Found existing installation: ansible 2.1.0.0 Uninstalling ansible-2.1.0.0: Successfully uninstalled ansible-2.1.0.0 Successfully installed ansible-2.2.1.0 paramiko-2.1.1
うまくいった。
Amazon Linuxで残りディスク容量アラートを設定
メトリクス送信設定
CloudWatchのメトリクスをLinuxから送信して、ディスク容量が減ったらアラートメールを送信するようにする。
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mon-scripts.html
http://docs.aws.amazon.com/ja_jp/AWSEC2/latest/UserGuide/mon-scripts.html
https://aws.amazon.com/code/8720044071969977
設定対象のLinuxで作業
Metrics通知ツール本体の前に、使っているライブラリ?をインストール。
sudo yum install perl-Switch perl-DateTime perl-Sys-Syslog perl-LWP-Protocol-https
本体をダウンロード。
mkdir ~/cloudwatch/ cd ~/cloudwatch/ curl http://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip -O
解凍して、aws-scripts-mon/ を /usr/local/cloudwatch/ の下に移動。
sudo mkdir /usr/local/cloudwatch sudo mv aws-scripts-mon/ /usr/local/cloudwatch/
IAMユーザーを作成して、アクセスキーを設定。
cd /usr/local/cloudwatch/aws-scripts-mon cp awscreds.template awscreds.conf vi awscreds.conf
IAM
IAMユーザーにアタッチしたポリシー。
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "cloudwatch:PutMetricData", "cloudwatch:GetMetricStatistics", "cloudwatch:ListMetrics", "ec2:DescribeTags" ], "Resource": [ "*" ] } ] }
↓だとダメだった。
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "cloudwatch:PutMetricData", "cloudwatch:GetMetricStatistics", "cloudwatch:ListMetrics", "ec2:DescribeTags" ], "Resource": [ "arn:aws:logs:*:*:*", "arn:aws:cloudwatch:*:*:*" ] } ] }
cronに設定
cronに設定前にコマンドラインから試す。
/usr/local/cloudwatch/aws-scripts-mon/mon-put-instance-data.pl --disk-path=/ --disk-space-util --disk-space-used --disk-space-avail
crontab -e
*/5 * * * * /usr/local/cloudwatch/aws-scripts-mon/mon-put-instance-data.pl --disk-path=/ --disk-space-util --disk-space-used --disk-space-avail --from-cron
以上でMetrics送信の設定は完了。
動いているかの確認は、AWS Management Console の CloudWatchで
Metrics > Linux System > Filesystem, InstanceId, MountPath
残りディスク容量が減った時のアラートメールは、AWS Consoleでやる。
Alarms mail
AWS Management Console の CloudWatchに行く(インスタンスが動いているリージョン)
Alarms > Create Alarm
Search Metrics
DiskSpaceUtilization
で検索。
チェックボックスをチェックして、Name, Description, >= 80
で作成。
メール送信の機能は AWS SNS の機能を使っているらしい。
メトリクスではなくてCloudWatchにログを出力(メモ)
CloudWatchにログを出力することもできるらしい。
この手順はMetrics送信の設定の前にやったけど、これじゃなかった。
この記事の通りにやった。
http://docs.aws.amazon.com/ja_jp/AmazonCloudWatch/latest/logs/QuickStartEC2Instance.html
2017-02-06 02:06:45,273 - cwlogs.threads - ERROR - 12442 - Thread-23 - Exception caught in <EventBatchPublisher(Thread-23, started daemon 139840766506752)> Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/cwlogs/threads.py", line 58, in run self._run() File "/usr/lib/python2.7/dist-packages/cwlogs/push.py", line 1391, in _run self._publish_event_batch() File "/usr/lib/python2.7/dist-packages/cwlogs/push.py", line 1198, in _publish_event_batch self.sequence_token = self._put_log_events(self.event_batch) File "/usr/lib/python2.7/dist-packages/cwlogs/push.py", line 1236, in _put_log_events response = self.logs_service.put_log_events(**params) File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 159, in _api_call return self._make_api_call(operation_name, kwargs) File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 483, in _make_api_call operation_model, request_dict) File "/usr/lib/python2.7/dist-packages/botocore/endpoint.py", line 117, in make_request return self._send_request(request_dict, operation_model) File "/usr/lib/python2.7/dist-packages/botocore/endpoint.py", line 142, in _send_request request = self.create_request(request_dict, operation_model) File "/usr/lib/python2.7/dist-packages/botocore/endpoint.py", line 126, in create_request operation_name=operation_model.name) File "/usr/lib/python2.7/dist-packages/botocore/hooks.py", line 227, in emit return self._emit(event_name, kwargs) File "/usr/lib/python2.7/dist-packages/botocore/hooks.py", line 210, in _emit response = handler(**kwargs) File "/usr/lib/python2.7/dist-packages/botocore/signers.py", line 90, in handler return self.sign(operation_name, request) File "/usr/lib/python2.7/dist-packages/botocore/signers.py", line 147, in sign auth.add_auth(request) File "/usr/lib/python2.7/dist-packages/botocore/auth.py", line 314, in add_auth raise NoCredentialsError NoCredentialsError: Unable to locate credentials
sudo vi /var/log/awslogs.log
を書き換えてリスタート
sudo service awslogs restart
2017-02-06 02:10:23,622 - cwlogs.push.reader - INFO - 12842 - Thread-4 - Start reading file from 0. 2017-02-06 02:10:25,109 - cwlogs.push.publisher - WARNING - 12842 - Thread-2 - Caught exception: An error occurred (AccessDeniedException) when calling the PutLogEvents operation: User: arn:aws:iam::9915xxxxxxxxxx:user/mmr2host-awslogs is not authorized to perform: logs:PutLogEvents on resource: arn:aws:logs:us-east-1:9915xxxxxxxxxx:log-group:/var/log/messages:log-stream:i-5ca0f181 2017-02-06 02:10:25,110 - cwlogs.threads - ERROR - 12842 - Thread-2 - Exception caught in <EventBatchPublisher(Thread-2, started daemon 139899077621504)> Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/cwlogs/threads.py", line 58, in run self._run() File "/usr/lib/python2.7/dist-packages/cwlogs/push.py", line 1391, in _run self._publish_event_batch() File "/usr/lib/python2.7/dist-packages/cwlogs/push.py", line 1198, in _publish_event_batch self.sequence_token = self._put_log_events(self.event_batch) File "/usr/lib/python2.7/dist-packages/cwlogs/push.py", line 1236, in _put_log_events response = self.logs_service.put_log_events(**params) File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 159, in _api_call return self._make_api_call(operation_name, kwargs) File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 494, in _make_api_call raise ClientError(parsed_response, operation_name) ClientError: An error occurred (AccessDeniedException) when calling the PutLogEvents operation: User: arn:aws:iam::9915xxxxxxxxxx:user/mmr2host-awslogs is not authorized to perform: logs:PutLogEvents on resource: arn:aws:logs:us-east-1:9915xxxxxxxxxx:log-group:/var/log/messages:log-stream:i-5ca0f181
2017-02-06 02:21:13,763 - cwlogs.push.reader - INFO - 12842 - Thread-24 - Replay events end at 14798. 2017-02-06 02:21:13,763 - cwlogs.push.reader - INFO - 12842 - Thread-24 - Start reading file from 0. 2017-02-06 02:21:14,890 - cwlogs.push.publisher - WARNING - 12842 - Thread-23 - Caught exception: An error occurred (ResourceNotFoundException) when calling the PutLogEvents operation: The specified log group does not exist. 2017-02-06 02:21:14,890 - cwlogs.push.batch - INFO - 12842 - Thread-23 - Creating log group /var/log/messages. 2017-02-06 02:21:15,081 - cwlogs.push.batch - INFO - 12842 - Thread-23 - Creating log stream i-5ca0f181. 2017-02-06 02:21:15,316 - cwlogs.push.publisher - INFO - 12842 - Thread-23 - Log group: /var/log/messages, log stream: i-5ca0f181, queue size: 0, Publish batch: {'skipped_events_count': 0, 'first_event': {'timestamp': 1486264861000, 'start_position': 0L, 'end_position': 151L}, 'fallback_events_count': 0, 'last_event': {'timestamp': 1486345988000, 'start_position': 14706L, 'end_position': 14798L}, 'source_id': '9eb733a4a488d353c22b89549724bc1e', 'num_of_events': 159, 'batch_size_in_bytes': 18773} 2017-02-06 02:21:19,437 - cwlogs.push.publisher - INFO - 12842 - Thread-23 - Log group: /var/log/messages, log stream: i-5ca0f181, queue size: 0, Publish batch: {'skipped_events_count': 0, 'first_event': {'timestamp': 1486347485000, 'start_position': 14798L, 'end_position': 14900L}, 'fallback_events_count': 0, 'last_event': {'timestamp': 1486347485000, 'start_position': 14984L, 'end_position': 15076L}, 'source_id': '9eb733a4a488d353c22b89549724bc1e', 'num_of_events': 3, 'batch_size_in_bytes': 353}
ログの出力場所
正しく設定できているはずなのに、CloudWatchに出てこないと思ったら、
us-east-1 (US East (N. Virginia)) に出来ていた。
監視対象は us-west-2 (US West (Oregon))にあるのに。
yumで入れた設定ファイルの初期値で region = us-east-1 と書いてある。
[plugins] cwlogs = cwlogs [default] region = us-east-1
PostgreSQLのリプリケーションをしてみるテスト
準備
postgresql94
AWS上にt1.smallでPostgresqlが動いているAmazon Linuxを2台構築。
Amazon Linux
$ cat /etc/os-release NAME="Amazon Linux AMI" VERSION="2016.09" ID="amzn" ID_LIKE="rhel fedora" VERSION_ID="2016.09" PRETTY_NAME="Amazon Linux AMI 2016.09" ANSI_COLOR="0;33" CPE_NAME="cpe:/o:amazon:linux:2016.09:ga" HOME_URL="http://aws.amazon.com/amazon-linux-ami/"
参考にしたウェブサイト
PostgreSQL レプリケーション構成を組んでみる - tom__bo’s Blog
PostgreSQL9.4でストリーミングレプリケーションする - マコーの日記
エラー1
-bash-4.2$ pg_basebackup -h 172.40.24.192 -p 5432 -D /var/lib/pgsql94/data --xlog --progress --verbose pg_basebackup: could not connect to server: could not connect to server: Connection timed out Is the server running on host "172.40.24.192" and accepting TCP/IP connections on port 5432?
AWS の security group に自分自身のsgを許可(5432)
エラー2
-bash-4.2$ pg_basebackup -h 172.40.24.192 -p 5432 -D /var/lib/pgsql94/data --xlog --progress --verbose pg_basebackup: could not connect to server: FATAL: no pg_hba.conf entry for replication connection from host "172.40.135.122", user "postgres", SSL off
Postgresql doesn't accept replication connection - Database Administrators Stack Exchange
-bash-4.2$ diff pg_hba.conf pg_hba.conf.2017203 85,86d84 < host all all 172.40.135.122/32 trust < host replication postgres 172.40.135.122/32 trust
pg_basebackup が動いた。
-bash-4.2$ pg_basebackup -h 172.40.24.192 -p 5432 -D /var/lib/pgsql94/data --xlog --progress --verbose transaction log start point: 0/E000028 on timeline 1 51646/51646 kB (100%), 1/1 tablespace transaction log end point: 0/E000128 pg_basebackup: base backup completed
メモ
Amazon Linux の場合、
recovery.conf.sample
は
/usr/share/pgsql94/recovery.conf.sample
にあった。
エラー3
INSERT文が止まる。
アプリも動かない(nginx 504 error: Gateway Timeout)
app1=> select application_name, client_addr, backend_start, state, sync_state from pg_stat_replication; application_name | client_addr | backend_start | state | sync_state ------------------+-------------+---------------+-------+------------ slave01 | | | | (1 row)
参考にしたウェブサイトと違って client_addr や state などが空になっていた。
ただ、この状態はエラーの原因ではなかった。
psql から INSERTして応答がなかったので、controll + c した時のメッセージ。
Cancel request sent WARNING: canceling wait for synchronous replication due to user request DETAIL: The transaction has already committed locally, but might not have been replicated to the standby. INSERT 0 1
このエラーは、
master db の postgresql.conf に書く、synchronous_standby_names の値と
slave db の recovery.conf に書く、primary_conninfo の中の値が違っていると発生するっぽい。
レコード自体は作られていた。
postgresql.conf in master db. synchronous_standby_names = 'slave01' # standby servers that provide sync rep recovery.conf in slave db. primary_conninfo = 'host=172.40.24.192 port=5432 application_name=slave01'
cronで作成日時から一定期間過ぎたファイルを削除
メモ:
cronで作成日時から一定期間過ぎたファイルを削除
MAILTO="" # delete files older than 3 days # 1 3,15 * * * find /var/www/app1/shared/docs/ -maxdepth 2 -type f -ctime +3 -delete 1 3,15 * * * find /var/www/app1/shared/docs/input/ -maxdepth 1 -type f -ctime +3 -delete 5 3,15 * * * find /var/www/app1/shared/docs/output/ -maxdepth 1 -type f -cmin +720 -delete
ansistranoのafter_symlinkでsudo(become)を実行する
ansible の YAML
ansible ansistrano の after_symlink hook で
/etc/init.d/
にシンボリックリンクを作る。
- name: initd symlink become: true file: state: link src: /var/www/app1/current/deploy/initd/app1_uwsgi #dest: /home/user1/app1_uwsgi dest: /etc/init.d/app1_uwsgi mode: "u=rwx,g=rx,o=rx"
sudoをするにはbecomeを使うと書いてあったけど、
指定の方法が分からなかった。
“配列の中にハッシュをネスト” という方法で書かないといけないっぽい。
http://docs.ansible.com/ansible/become.html
http://www.task-notes.com/entry/20150922/1442890800
ansible-playbook -i inventory/hosts/localvm playbook/deploy.yml -K
ansible-playbook コマンドの最後に -K を付けると最初にsudo passwordを聞かれる。
Ansible, sudoパスワード要求を忘れただけでめんどくなる - Goldstine研究所
エラーメッセージ
その1
fatal: [192.168.3.43]: FAILED! => {"failed": true, "reason": "no action detected in task. This often indicates a misspelled module name, or incorrect module path.\n\nThe error appears to have been in '/Users/aaa/PycharmProjects/app1/deploy/playbook/ansistrano/after_symlink.yml': line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- tasks:\n ^ here\n\n\nThe error appears to have been in '/Users/aaa/PycharmProjects/app1/deploy/playbook/ansistrano/after_symlink.yml': line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- tasks:\n ^ here\n"}
その2
fatal: [192.168.3.43]: FAILED! => {"failed": true, "reason": "Syntax Error while loading YAML.\n\n\nThe error appears to have been in '/Users/aaa/PycharmProjects/app1/deploy/playbook/ansistrano/after_symlink.yml': line 4, column 18, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n- name: create django static files\n django_manage:\n ^ here\n"}
完成品
PycharmProjects/app1/deploy/playbook/ansistrano/after_symlink.yml
- name: create django static files django_manage: command: collectstatic app_path: "{{ ansistrano_deploy_to }}/current/" virtualenv: "/home/user1/.pyenv/versions/app1/" - name: init.d app1_uwsgi symlink become: true file: state: link src: /var/www/app1/current/deploy/initd/app1_uwsgi #dest: /home/user1/app1_uwsgi dest: /etc/init.d/app1_uwsgi mode: "u=rwx,g=rx,o=rx" - name: init.d app1_celeryd symlink become: true file: state: link src: /var/www/app1/current/deploy/initd/app1_celeryd dest: /etc/init.d/app1_celeryd mode: "u=rwx,g=rx,o=rx" - name: restart app1_uwsgi become: true service: name: app1_uwsgi state: restarted - name: restart app1_celeryd become: true service: name: app1_celeryd state: restarted