[GLP] Play With Shell

Sun, 07 Jun 2020 11:33:39 +0800

Two Efficient Methods for Line-by-Line File Processing in Shell

Method 1: Using a File Descriptor

Redirect stdout to a file descriptor (fd 4), then restore it after processing. This is slightly faster for large files.

function while_read_line_bottom_fd_out
{
  >$OUTFILE
  exec 4<&1
  exec 1>$OUTFILE
  while read LINE
  do
    echo "$LINE"
    :
  done < $INFILE
  exec 1<&4
  exec 4>&-
}

Method 2: Without a File Descriptor

Simpler and easier to maintain — appends each line directly to the output file.

Crontab Issue Tracking

Mon, 07 Aug 2017 12:33:21 +0800

how to track issue

`1`	`sudo /var/log/syslog \| grep cron`

capture the output

`1`	`1 2 * * * /home/hyy/Start.py >/tmp/output.log 2>&1`

check cron is running

`1`	`ps -ef \| grep cron \| grep -v grep`

check the path is correct

crontab is running in with cwd == $HOME, if you using python os.getcwd() in different folder structure, make sure you cd to the correct place or else it will affect your code by output “can not find module” stuff.
in short, use absolute path in your command or add source ~/.zshrc before your crontab script, so it can find the environment variable

last command in crontab should have a blank line

this is tricky, if you didn’t notice it.

be careful the dot thing

Debian Linux and its derivative (Ubuntu, Mint, etc) have some peculiarities that may prevent your cron jobs from executing; in particular, the files in /etc/cron.d, /etc/cron.{hourly,daily,weekly,monthly} must :
- be owned by root
- only be writable by root
- not be writable by group or other users
- have a name without any dots ‘.’ or any other special character but ‘-’ and ‘_’ .
python issue run in crontab
make sure python script have #!/usr/bin/python at the beginning, not neccessary
make sure log the output, so you can debug in a smooth way

chmod to the script

always run in normal user mode, not root user, and do not use
1 2

# don't do this sudo contab -e
if you try to put your log in other file, make sure the perssion of the file is good, some times, guys delete the log file via root user and create a new one with root user, this will cause the permission error.the right way is echo "" > logfile, then you won’t change the original file’s permission and group

Shell on H&W