Listing posts
Displaying posts 1 - 5 of 333 in total2024-09-19
Resources
Extensions
-
- anicode find arbitrary unicode characters matching a search pattern
autoenvstack folder custom env set/restore- cprintf printf with colors
g2 friendly git command line client- getopts command line options parser
- gitignore create .gitignore files from https://www.gitignore.io templates
- paths create or append to environment variables
rvm rvm support- shark sparkline generator (cli histograms)
- spin background job spinner
termcolours list available terminal colour names- upto change to the closest parent folder by name
oh-my-fish plugin manager (list)
prompt themes (see powerline and nerd fonts):
- oh-my-fish bobthefish: powerline-style, git-aware theme
- fisher metro powerline prompt optimized for speed and space
- fisher nitro color-smart powerline prompt based in Metro
- fisher segment powerline prompt builder
Prompt power-up
- decompress
fish_config.7z
attachment in~/.config/fish/
curl -L https://get.oh-my.fish | fish
omf install bobthefish
- download and install
Meslo
font from nerd-fonts - set
MesloLGMDZ Nerd Font Mono
font for your terminal - put
set -g theme_nerd_fonts yes
in your~/.config/fish/config.fish
- customize
fish_greeting.fish
andfish_right_prompt.fish
in~/.config/fish/functions/
On cygwin under windows you can install DejaVu Sans Mono
ttf powerline font, and set fish as the default shell.
In the attached file there is a simple prompt with git support.
Commands
fish_update_completions
-- scan man pages for completion tipsenv VAR1=xxx VAR2=yyy command arg1 arg2 ...
-- run command with env vars
~~~ * ~~~
2024-09-17
Installation on debian
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # check system compatibility modprobe configs # loads /proc/config.gz wget -q -O - https://raw.githubusercontent.com/docker/docker/master/contrib/check-config.sh | \ bash | tee docker-check.txt # install docker: key, repo, packages apt-get install apt-transport-https ca-certificates curl gnupg2 software-properties-common curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add - # amd64 - x64 echo "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable" > /etc/apt/sources.list.d/docker-ce.list # armhf - x32 / raspberry pi / raspbian echo "deb [arch=armhf] https://download.docker.com/linux/raspbian $(lsb_release -cs) stable" > /etc/apt/sources.list.d/docker-ce.list apt-get update && apt-get install docker-ce # allow user to use docker usermod -aG docker username # test installation docker version docker info # run a simple test image docker run hello-world |
See also post install
for troubleshooting dns/network/remote access.
On raspberry pi just use curl -sSL https://get.docker.com | sh
(repo not working).
Configure daemon
- change docker data folder location
1 2 3 4 5 | mkdir -p /path/to/data chown root.root /path/to/data chmod 711 /path/to/data echo '{ "data-root": "/path/to/data" }' > /etc/docker/daemon.json systemctl restart docker |
1 | echo '{ "log-driver": "local" }' > /etc/docker/daemon.json |
Creating an image (ref, best practices)
1 2 3 4 5 6 7 8 9 10 11 12 | touch Dockerfile # and fill it docker build -t test-myimg . # create the image with a tag # test run image docker run -p 4000:80 test-myimg docker run -it test-myimg /bin/bash # run image detached/on background docker run -p 4000:80 -d --name tmi test-myimg docker container ls -a docker container stop <container_id> docker container start -i tmi # restart container |
Interact (ref)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # run interactive shell into debian image (temporary) docker run --name prova --rm -it debian /bin/bash # run interactive shell into debian image docker run -it debian /bin/bash apt-get update apt-get install -y dialog nano ncdu apt-get install -y locales localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8 echo "LANG=en_US.utf8" >> /etc/environment rm -rf /var/lib/apt/lists/* docker commit e2b7329257ba myimg:v1 docker run --rm -it myimg:v1 /bin/bash # run a command in a running container docker exec -ti a123098734e bash -il docker stop a123098734e docker kill a123098734e |
Save & restore
1 2 3 4 5 6 7 8 9 10 | # dump image docker save imgname | gzip > imgname.tgz zcat imgname.tgz | docker load # dump container docker create --name=mytemp imgname docker export mytemp | gzip > imgname-container.tgz # flatten image layers (losing Dockerfile) from a container docker export <id> | docker import - imgname:tag |
Registry - Image repository
1 2 3 4 5 | # push image to gitlab registry docker login registry.gitlab.com docker tag test-myimg registry.gitlab.com/username/repo:tag # add new tag... docker rmi test-myimg # ...and remove the old tag docker push registry.gitlab.com/username/repo:tag |
Tips
1 2 3 | # remove untagged image -- https://stackoverflow.com/a/33913711/13231285
docker images --digests
docker image rm image-name@sha256:xxxxxxxxxxxxxxxxxxxxxxxxxx
|
DockerHub official base images links: debian, ruby, rails, redis, nginx.
Available free registry services:
Name | # Priv/Pub | Notes |
---|---|---|
gitlab | inf/ND | 1 prj x registry |
treescale | inf/inf | max 500 pulls & 50GB |
canister | 20/ND | very good service |
docker hub | 1/inf | perfect |
Running arm
image on x86
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | # https://ownyourbits.com/2018/06/27/running-and-building-arm-docker-containers-in-x86/ apt-get install qemu-user-static docker run \ -v /usr/bin/qemu-arm-static:/usr/bin/qemu-arm-static \ -e LANG=en_US.utf8 -ti --name myarmimg arm32v7/debian:wheezy [...] docker commit myarmimg myarmimg docker container prune -f docker run \ -v /usr/bin/qemu-arm-static:/usr/bin/qemu-arm-static \ -ti --rm --name myarmimg \ myarmimg /bin/bash -il |
Composer (ref, dl) - Services
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | # docker-compose.yml version: "3" services: web: image: username/repo:tag deploy: replicas: 5 resources: limits: cpus: "0.1" memory: 50M restart_policy: condition: on-failure ports: - "4000:80" networks: - webnet networks: webnet: |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # install docker-compose curl -L -o /usr/local/bin/docker-compose https://github.com/docker/compose/releases/download/1.24.0-rc1/docker-compose-`uname -s`-`uname -m` chmod 755 /usr/local/bin/docker-compose docker swarm init docker stack deploy --with-registry-auth -c docker-compose.yml getstartedlab docker service ls docker service ps getstartedlab_web # or docker stack ps getstartedlab # change the yml file and restart service docker stack deploy --with-registry-auth -c docker-compose.yml getstartedlab docker service ps getstartedlab_web docker container prune -f # stop & destroy service docker stack rm getstartedlab docker container prune -f # leave the swarm docker swarm leave --force |
Machine (ref, dl) - SWARM/Provisioning
Remember to update the host firewall: open port 2376
and do not apply rate limits on port 22
.
On the fish shell you can install the useful omf plugin-docker-machine to easily select the current machine.
Without an official supported driver we can use the generic one. Install docker-ce on your worker nodes and then in your swarm manager host:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 | # install docker-machine curl -L -o /usr/local/bin/docker-machine https://github.com/docker/machine/releases/download/v0.16.1/docker-machine-`uname -s`-`uname -m` chmod 755 /usr/local/bin/docker-machine # setup each VMs (this creates and shares the certificates for a secure # connetion between your client and the daemon runnig on the server) ssh-copy-id -i ~/.ssh/id_rsa user@ww.xx.yy.zz docker-machine create --driver generic --generic-ssh-key ~/.ssh/id_rsa \ --generic-ip-address=ww.xx.yy.zz myvm1 ssh-copy-id -i ~/.ssh/id_rsa user@ww.xx.yy.kk docker-machine create --driver generic --generic-ssh-key ~/.ssh/id_rsa \ --generic-ip-address=ww.xx.yy.kk myvm2 docker-machine ls # run a command via ssh in a VM docker-machine ssh myvm1 "ls -l" # use internal SSH lib docker-machine --native-ssh ssh myvm1 "bash -il" # use system SSH lib # set env to run all docker commands remotely on a VM eval $(docker-machine env myvm1) # on bash docker-machine use myvm1 # on fish + omf plugin-docker-machine # set VM1 to be a swarm manager docker-machine use myvm1 docker swarm init # --advertise-addr ww.xx.yy.zz docker swarm join-token worker # get token for adding worker nodes # set VM2 to join the swarm as a worker docker-machine use myvm2 docker swarm join --token SWMTKN-xxx ww.xx.yy.zz:2377 # check cluster status on your local machine... docker-machine ls # ...or on the manager node docker-machine use myvm1 docker node ls # locally login on your registry... docker-machine unset docker login registry.gitlab.com # ...then deploy the app on the swarm manager docker-machine use myvm1 docker stack deploy --with-registry-auth -c docker-compose.yml getstartedlab docker service ls docker service ps getstartedlab_web # access cluster from any VM's IP curl http://ww.xx.yy.zz:4000 curl http://ww.xx.yy.kk:4000 # eventually re-run "docker stack deploy ..." to apply changes # undo app deployment docker-machine use myvm1 docker stack rm getstartedlab # remove the swarm docker-machine ssh myvm2 "docker swarm leave" docker-machine ssh myvm1 "docker swarm leave --force" |
Stack / Deploy application
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | # docker-compose.yml version: "3" services: web: image: username/repo:tag deploy: replicas: 5 restart_policy: condition: on-failure resources: limits: cpus: "0.1" memory: 50M ports: - "80:80" networks: - webnet visualizer: image: dockersamples/visualizer:stable ports: - "8080:8080" volumes: - "/var/run/docker.sock:/var/run/docker.sock" deploy: placement: constraints: [node.role == manager] networks: - webnet redis: image: redis ports: - "6379:6379" volumes: - "/home/docker/data:/data" deploy: placement: constraints: [node.role == manager] command: redis-server --appendonly yes networks: - webnet networks: webnet: |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | docker-machine use myvm1 docker-machine ssh myvm1 "mkdir ./data" # create redis data folder # run stack / deploy app docker stack deploy -c docker-compose.yml getstartedlab docker stack ps getstartedlab # show deployed services and restart one docker service ls docker service update --force getstartedlab_web firefox http://<myvm1-ip>:8080/ # docker visualizer redis-cli -h <myvm1-ip> # interact with redis docker stack rm getstartedlab |
Init process to reap zombies and forward signals
- single process: tini (use
docker run --init
orinit: true
in docker-compose.yml) - multiprocess: s6 and s6-overlay
- init systems comparison
SWARM managers
- traefik: github, hp
- portainer (formerly ui-for-docker)
- swarmpit
- dry (terminal gui, one man prj)
- guides, tips and hints at dockerswarm.rocks (also on github)
Container-Host user remapping
You can map container users to the host ones for greater security.
- put
myuser:100000:65536
(start:length) in/etc/subuid
and/etc/subgid
, this defines the mapping id range 100000-165535 available to the host usermyuser
configure docker daemon to use the remapping specified for
myuser
:1 2
echo '{ "userns-remap": "myuser" }' > daemon.json systemctl restart docker
note that all images will reside in a /var/lib/docker subfolder named after
myuser
idsnow all your container user/group ids will be mapped to
100000+id
on the host
You can write up to 5 ranges in sub* files for each user, in this example we set identical ids for users 0-999 and map ids >=1000 to id+1:
1 2 | myuser:0:1000 myuser:1001:65536 |
UFW Firewall interactions
Docker bypasses UFW rules and published ports can be accessed from outside.
See a solution involving DOCKER-USER and ufw-user-forward/ufw-user-input chains.
Dockerizing Rails
- docker-rails-base -- preinstalled gems, multi stage, multi image, uses onbuild triggers
- dockerfile-rails -- Dockerfile extracted from Rails 7.1 by fly.io
- Kamal -- formerly MRSK, DHH solution, deploy web apps anywhere with zero downtime, guide posts
Terms:
service
= containers that only runs one/same image,task
= a single container running in a service,swarm
= a cluster of machines running Docker,stack
= a group of interrelated services orchestrated and scalable, defining and coordinating the functionality of an entire application.
Source: install, install@raspi, tutorial, overview, manage app data, config. daemon, config. containers,
Source for user mapping: docker docs, jujens.eu, ilya-bystrov
Useful tips: cleanup,
network host mode for nginx to get client real IP, limit ram/cpu usage, docker system prune -a -f
to remove all cache files
See also: thread swarm gui, docker swarm rocks
~~~ * ~~~
2024-09-16
Name | IPs | Origin | Notes |
---|---|---|---|
DNS.watch | 84.200.69.80 84.200.70.40 |
EU / Munich, Germany | no logs, DNSSEC |
FreeDNS | 37.235.1.174 37.235.1.177 |
EU / Wien, Austria | no logs |
Cloudflare | 1.1.1.1 1.0.0.1 1dot1dot1dot1.cloudflare-dns.com |
HK / Honk Kong AU / Australia |
logs lasts 24h does not log IPs |
DNScrypt.eu | 77.66.84.233:443 | EU / Koebenhavn, Denmark | DNSCrypt only1 |
DNScrypt.eu | 176.56.237.171:443 | EU / Nuland, Netherlands | DNSCrypt only1 |
UncensoredDNS | 91.239.100.100 89.233.43.71 |
EU / Copenhagen, Denmark | no logs, AnyCast |
OpenNIC | many | * | depends on server |
Source: greycoder
Android app: OverrideDNS
-
DNSCrypt: DNSCrypt.eu usage, wikipedia, dnscrypt-autoinstall, dnscrypt-proxy ↩
~~~ * ~~~
2024-09-12
apps/addons/plugins fix»
Add-ons
- dark reader
- DeepL Translate -- store
- keepa amazon tracker
- tampermonkey
- user agent switcher
- video downloadhelper -- hp
developer mode
Go to chrome://extensions/ / vivaldi://extensions/ and toggle developer mode switch.
Vivaldi
Command line options
1 2 3 4 5 6 7 8 9 | # https://www.ghacks.net/2017/02/13/how-to-speed-up-the-vivaldi-web-browser/ # optimized command for raspberry pi /usr/bin/vivaldi \ --process-per-site \ --enable-low-res-tiling \ --enable-low-end-device-mode \ --disk-cache-size=104857600 \ --disk-cache-dir=$TMPD \ "$@" |
Delete undeletable cookies
If you can't delete them from Settings > Privacy and Security > Cookies:
- Open Site Settings (click on the lock in the address field and selecting the last menu item)
- Go back and filter the desired site
- Delete cookies and data storage
Fix passwords not syncing
If you observe the following error in vivaldi://sync
1 2 | Error: CleanupPasswordStore@components/password_manager/core/browser/sync/password_sync_bridge.cc:1067, datatype error was encountered: Failed to get encryption key during database cleanup. |
- close Vivaldi
rm -f ~/.config/vivaldi/Default/Login\ Data*
- launch Vivaldi and the sync error in vivaldi://sync should have vanished
Video DRM/Widevine on Vivaldi
1 | find ~/.config/vivaldi -type d -regex ".+\(GPU\|Graphite\|Shader\|Dawn\)Cache" -exec rm -rf "{}" + |
obsolete instructions
- vivaldi://flags > Override software rendering list > Enable
- vivaldi://gpu
- on Rpi5 disable chrome://flags/#disable-accelerated-video-decode
- vivaldi://components > Widevine Content Decryption Module > check if it's loaded
See: old script, vivaldi forum, test video, another test page
1 2 | apt install libwidevinecdm0 echo '{"Path":"/opt/WidevineCdm"}' > ~/.config/vivaldi/WidevineCdm/latest-component-updated-widevine-cdm |
~~~ * ~~~
2024-09-05
MOGLIE / ANDROID
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 | # ----- FOTO ------------------------------------------------------------------- # resize f.jpg to _f.jpg ruby -e '%w{fileutils shellwords}.each{|l| require l}; lista=Dir["**/*"].grep(/\.jpg/i).sort; lista.each_with_index{|f,i| '\ 'puts "#{((i+1)/lista.size.to_f*100).to_i}%\t#{f}"; next if File.basename(f).start_with?("_"); system %Q[ '\ 'vips jpegsave #{f.shellescape} /tmp/rz.jpg --Q 80 && '\ 'jhead -autorot /tmp/rz.jpg && '\ 'mv /tmp/rz.jpg #{File.dirname(f).shellescape}/_#{File.basename(f).shellescape} && '\ 'rm -f #{f.shellescape} || echo ERRORE ] }' # rename by exif/file date ruby -e 'lista=Dir["**/*"].grep(/\.jpg/i).sort; lista.each_with_index{|f,i| '\ 'print "#{((i+1)/lista.size.to_f*100).to_i}%\t"; '\ 'system %Q[jhead -n"%Y-%m-%d_%H-%M-%S" "#{f}"] }' ## OR rename _f.jpg to f.jpg #ruby -e 'lista=Dir["**/*"].grep(/\.jpg/i).sort; lista.each_with_index{|f,i| '\ #'puts "#{((i+1)/lista.size.to_f*100).to_i}%\t#{f}"; next unless File.basename(f).start_with?("_"); '\ #'File.rename(f, File.join(File.dirname(f), File.basename(f).sub(/^_/, ""))) }' # crea sottocartelle foto e video (per ogni cartella avente files video) ruby -e 'Dir["**/*"].grep_v(/\.jpg/i).map{|f| File.dirname(f) if File.file?(f) }.compact.sort.uniq.each{|d| '\ 'Dir.chdir(d){ files = Dir["*"]; system "mkdir -p foto video"; '\ 'files.each{|f| dst = (f =~ /\.jpg/i ? :foto : :video); File.rename(f, "#{dst}/#{f}") } } }' # zippa cartelle di sole foto ruby -e 'require "shellwords"; Dir["**/*"].grep(/\.jpg/i).map{|f| File.dirname(f) }.compact.sort.uniq.each{|d| '\ 'puts d; Dir.chdir(File.join d, ".."){ n=File.basename(d); system "zip -r #{n.shellescape}.zip #{n.shellescape} && rm -rf #{n.shellescape}" } }' # ----- AUDIO ------------------------------------------------------------------ # converti video neri in MP3 => ffmpeg -i in.mp4 -ab 128k out.mp3 mdkir audio && cd audio for i in *.mp4; ffmpeg -i $i -ab 128k (string replace mp4 mp3 $i); end # fish #for i in *.mp4; do ffmpeg -i "$i" -ab 128k "${i%.*}.mp3"; done # bash rm -f *.mp4 # ----- VIDEO ------------------------------------------------------------------ # converti video MP4 in AVI/h264 for i in (find -type f -iname "*.mp4" -printf '%s\t%p\n' | sort -n | cut -f 2) echo $i vid2all -l -v 2048 -a 192 --no-expand --no-resize $i /mnt/ramd/out.avi \ && mv /mnt/ramd/out.avi (string replace -i mp4 avi $i) \ || touch (string replace -i mp4 ERR $i) end find -type f -name "*.ERR" # sposta tutti gli MP4 for i in (find -type f -iname "*.mp4"|sed -r 's/^..//'); mv -i $i (echo $i|sed -r 's/ /_/g ; s/\//--/g'); end |
SUOCERI / WINDOWS
external tools
- ffmpeg for windows
- jhead
- VisiPics photo duplicate finder
- WinDirStat folder tree size
comprimi_media
1 2 3 4 5 6 7 8 9 10 11 | #!/bin/env ruby dirs = %x[ find -mindepth 1 -maxdepth 1 -type d ].split("\n").sort dirs.each do |d| puts "===== CARTELLA [ #{d} ] =====" print 'foto : '; system 'ls|egrep -i "(jpe*g|png|heic)$"|wc -l', chdir: d print 'video: '; system 'ls|egrep -i "(mp4|mov|avi|mpeg|hevc)$"|wc -l', chdir: d system 'comprimi_media.single', chdir: d end puts "\n\nFINE" |
comprimi_media.single
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | #!/bin/env ruby %w{ shellwords FileUtils progressbar }.each{|l| require l } pb_opts = { progress_mark: '#', remainder_mark: '_', length: 79, starting_at: 0, format: '%t: %J |
misc ruby
1 2 3 4 5 | # rename_foto_by_ts ruby -e 'Dir["**/*"].grep(/\.jpg/i).sort.each{|f| system %Q[jhead -n"%Y-%m-%d_%H-%M-%S" "#{f}"] }' # enumerate files Dir['*'].sort.each_with_index{|f, i| File.rename f, "#{'%04d' % (i+1)}.mp4"}; nil |
misc shell
1 2 3 4 5 | # PDF to JPG ls *.pdf | sed -r 's/(.+).pdf/pdftoppm -jpeg \0 \L\1.jpg/' | sh # lossless rotate JPG jpegtran -rotate 90 in.jpg > out.jpg |