Full Blog TOC

Full Blog Table Of Content with Keywords Available HERE

Monday, December 18, 2023

List Files with Epoch Names



In many applications and solutions we create files with timestamp, and a very common method is to use epoch seconds in the name of the files. This is very useful to automated analysis of the files. But, from time to time, we might need to manually analyze the files, and then locating the file with the epoch we want is is tough task. Consider the following files list:


create_time.txt
graphs1702167500-1702171090.json
graphs1702171100-1702174690.json
graphs1702171100-a.json
graphs1702174700-1702178290.json
graphs1702178300-1702181890.json
graphs1702181900-1702185490.json
graphs1702185500-1702189090.json
graphs1702189100-1702192690.json
graphs1702192700-1702196290.json
graphs1702196300-1702199890.json
graphs1702199900-1702203490.json
graphs1702203500-1702205010.json
signatures_1702167500-1702167500.json
signatures_1702167510-1702167510.json
signatures_1702167520-1702167520.json
signatures_1702167530-1702167530.json
signatures_1702167540-1702167540.json
signatures_1702167550-1702167550.json
signatures_1702167560-1702167560.json
signatures_1702167570-1702167570.json
signatures_1702167580-1702167580.json


we can copy the epoch from the file name, and use bash's `date` command to find the actual time of the file, for example:


$ date -u -d @1702171100
Sun 10 Dec 2023 01:18:20 UTC


But doing this for each file is not fun. Instead, we can use a script to automate listing of the files. The following script does the job:


#!/usr/bin/env bash
set -e

folder=$1

ProcessLine() {
file=$1
timeDescription="missing epoch"
epoch1=$(echo "$file" | sed 's/[a-zA-Z_-]*\([0-9]\{10\}\).*/\1/p' | head -1)
epoch1Length=${#epoch1}
if [[ "${epoch1Length}" == "10" ]]; then
date1=$(date +%Y-%m-%dT%H:%M:%S -u -d @${epoch1})
epoch2=$(echo "$file" | sed 's/.*\([0-9]\{10\}\).*\([0-9]\{10\}\).*/\2/p' | head -1)
epoch2Length=${#epoch2}
if [[ "${epoch2Length}" == "10" ]]; then
date2=$(date +%Y-%m-%dT%H:%M:%S -u -d @${epoch2})
timeDescription="${date1} to ${date2}"
else
timeDescription="${date1}"
fi
fi

sizeDescription=$(ls -lh ${file} | awk '{print $5}')
sizeDescription=$(printf "%8s %s" ${sizeDescription})
fileDescription=$(printf "%-50s %s" ${file})

echo "${fileDescription} ${sizeDescription} ${timeDescription}"
}

ls $1 | while read line; do ProcessLine "$line"; done


The output from the script lists all files in the current working directory, including the file name, size, and user friendly times. It supports files without epoch in the name, files with a single epoch in the name, and files with epoch range in their names. An example output is:


create_time.txt                                           51  missing epoch
graphs1702167500-1702171090.json 193K 2023-12-10T00:18:20 to 2023-12-10T01:18:10
graphs1702171100-1702174690.json 193K 2023-12-10T01:18:20 to 2023-12-10T02:18:10
graphs1702171100-a.json 0 2023-12-10T01:18:20
graphs1702174700-1702178290.json 193K 2023-12-10T02:18:20 to 2023-12-10T03:18:10
graphs1702178300-1702181890.json 181K 2023-12-10T03:18:20 to 2023-12-10T04:18:10
graphs1702181900-1702185490.json 184K 2023-12-10T04:18:20 to 2023-12-10T05:18:10
graphs1702185500-1702189090.json 194K 2023-12-10T05:18:20 to 2023-12-10T06:18:10
graphs1702189100-1702192690.json 195K 2023-12-10T06:18:20 to 2023-12-10T07:18:10
graphs1702192700-1702196290.json 195K 2023-12-10T07:18:20 to 2023-12-10T08:18:10
graphs1702196300-1702199890.json 178K 2023-12-10T08:18:20 to 2023-12-10T09:18:10
graphs1702199900-1702203490.json 190K 2023-12-10T09:18:20 to 2023-12-10T10:18:10
graphs1702203500-1702205010.json 76K 2023-12-10T10:18:20 to 2023-12-10T10:43:30
signatures_1702167500-1702167500.json 29 2023-12-10T00:18:20 to 2023-12-10T00:18:20
signatures_1702167510-1702167510.json 29 2023-12-10T00:18:30 to 2023-12-10T00:18:30
signatures_1702167520-1702167520.json 29 2023-12-10T00:18:40 to 2023-12-10T00:18:40
signatures_1702167530-1702167530.json 29 2023-12-10T00:18:50 to 2023-12-10T00:18:50
signatures_1702167540-1702167540.json 29 2023-12-10T00:19:00 to 2023-12-10T00:19:00
signatures_1702167550-1702167550.json 29 2023-12-10T00:19:10 to 2023-12-10T00:19:10







No comments:

Post a Comment