USGS Tetracorder Docker

Non esistono metodi banali per installare Tetracorder (versione moderna) ma una versione comoda e' costituita da un docker non ufficiale

 

git clone https://github.com/jl-0/tetracorder-lite.git

cd tetracorder-lite

 prima di fare la build si deve modificare il file Containerfile aggiungendo la riga evidenziata in giallo

RUN cd tetracorder &&\
    sed -i "398,416 s/^/#/" AAA.INSTALL.spectroscopy-os-setup-linux.sh &&\
    sed -i "231,254 s/^/#/" AAA.INSTALL.spectroscopy-os-setup-linux.sh &&\
    yes "y" | bash AAA.INSTALL.spectroscopy-os-setup-linux.sh install &&\
    cd tetracorder &&\
    chmod +x checkmultmapcompilemode &&\
    make install &&\
    sed -i "137,140 s/^/#/" multmap.h &&\
    sed -i "144,147 s/^#//" multmap.h &&\
    make installsingle 

a questo punto si fa la build

docker build --platform linux/amd64 -f Containerfile -t tetracorder-lite . 

e si creano dei folder di lavoro

mkdir -p ~/tetracorder/input ~/tetracorder/output 

per controllare che tutto sia a posto si puo' lanciare 

docker run --rm tetracorder-lite tetrapy --help
docker run --rm tetracorder-lite tetrapy run --help

a questo punto siamo pronti a lanciare una prima elaborazione.Il problema e' che  Tetracorder accetta immagini in formato Envi mentre le immagini Emit sono distribuite in formato NetCDF (si devono scarivare sia i dati RFL di riflettanza che RFLUNCERT dell'incertezza della riflettanza

Per la conversione si possono le emit utilities 

git clone https://github.com/nasa/emit-utils.git
cd emit-utils
python emit_utils/reformat.py /path/to/EMIT_L2A_RFL_xxx.nc /path/to/output_envi/ --orthorectify 

funzionano ma non sono riuscito ad installare su Debian pip install xarray netCDF4 rioxarray earthaccess ovvero i modulo per orthorectify...si puo' fare a meno per questa prova

cd ~/emit-utils
python -m emit_utils.reformat ./emit_utils/EMIT_L2A_RFL_001_20260825T085423_2623706_045.nc /home/luca/tetracorder/input/ --orthorectify

arrivati a questo punto si puo' lanciare il processing  

docker run --rm \
  -v ~/tetracorder/input:/data \
  -v ~/tetracorder/output:/output \
  tetracorder-lite \
  tetrapy run config.yml \
    --data.rfl /data/EMIT_L2A_RFL_001_20260825T085423_2623706_045_reflectance \
    --data.rfluncert /data/EMIT_L2A_RFLUNCERT_001_20260825T085423_2623706_045_reflectance_uncertainty 

 

al termine del processing avremo una serie di file tipo la seguente dove

group.1um, group.1.5um-broad, group.2um, group.2um-broad — minerali (argille, carbonati, solfati, ecc., a seconda della banda di assorbimento)

group.ree, group.ree_g21, group.ree_samar — terre rare (rare earth elements)

group.ch4gas-2.3um, group.co2gas-2um — gas (metano, CO2)

group.veg — vegetazione

case.ep-cal-chl — epidoto/calcite/clorite

case.red-edge — red edge della vegetazione

case.veg.type — tipologia di vegetazione

  

.
./test
./test/aggregate
./test/aggregate/agg.nc
./test/aggregate/config.yml
./test/aggregate/agg-uncert.nc
./test/aggregate/tetrapy.log
./test/tetracorder
./test/tetracorder/AAAA.license.txt
./test/tetracorder/group.1.5um-broad
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_vbroad_br20.fit.gz
./test/tetracorder/group.1.5um-broad/olivine_fo89_gds70.d.depth.gz
./test/tetracorder/group.1.5um-broad/snow-ice-10mm+0.3water-273k.depth.gz
./test/tetracorder/group.1.5um-broad/vegetation1.fit.gz
./test/tetracorder/group.1.5um-broad/snow-ice-3mm+0.2water-273k.depth.gz
./test/tetracorder/group.1.5um-broad/water.high.chlorophyll2.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/olivine_fo18_ki3377.fd.gz
./test/tetracorder/group.1.5um-broad/water-20C-100cm-path+chlrph-a.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/snow-ice-3mm+0.1water-273k.fit.gz
./test/tetracorder/group.1.5um-broad/water+mont300gpl.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/snow-ice-3mm+0.2water-273k.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/water+mont50gpl.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/olivine_fo60_ki3189.fit.gz
./test/tetracorder/group.1.5um-broad/water-20C-300cm-path+chlrph-g.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/water+mont0.5gpl.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/snow.melting16+0.5veg.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/anorthite_hs201.3b.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/water-20C-100cm-path+chlrph-a.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/water-20C-10cm-path-model.depth.gz
./test/tetracorder/group.1.5um-broad/water-20C-100cm-path-model.fit.gz
./test/tetracorder/group.1.5um-broad/snow.melting.8.fd.gz
./test/tetracorder/group.1.5um-broad/snow-ice-10mm-273k.fd.gz
./test/tetracorder/group.1.5um-broad/snow.slush.9.depth.gz
./test/tetracorder/group.1.5um-broad/water+mont0.5gpl.fd.gz
./test/tetracorder/group.1.5um-broad/olivine_fo66_ki3054.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/olivine_fo60_ki3189.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_brd.br22c_actinolite.fit.gz
./test/tetracorder/group.1.5um-broad/vegetation1.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/water+mont50gpl+0.5veg.fit.gz
./test/tetracorder/group.1.5um-broad/water-20C-30cm-path-model.fit.gz
./test/tetracorder/group.1.5um-broad/anorthite_hs201.3b.fd.gz
./test/tetracorder/group.1.5um-broad/snow-ice-3mm+0.3water-273k.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/snow-ice-3mm+0.1water-273k.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/rutile_hs126.3b.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/snow-ice-3mm+0.1water-273k.fd.gz
./test/tetracorder/group.1.5um-broad/water-20C-30cm-path-model.depth.gz
./test/tetracorder/group.1.5um-broad/snow.melting16+0.5veg.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_nrw.cummingtonite.fd.gz
./test/tetracorder/group.1.5um-broad/vegetation.dry+green.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/water-20C-300cm-path+chlrph-g.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/olivine_fo66_ki3054.depth.gz
./test/tetracorder/group.1.5um-broad/snow-ice-1mm-273k.fd.gz
./test/tetracorder/group.1.5um-broad/water+mont16.5gpl.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_chlorite.clinochlor.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/snow-ice-3mm-273k.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_vbroad_br20.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/water-20C-100cm-path+chlrph-g.fit.gz
./test/tetracorder/group.1.5um-broad/snow-ice-3mm+0.3water-273k.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/microcline_hs103.3b.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/snow-ice-1mm-273k.depth.gz
./test/tetracorder/group.1.5um-broad/snow.slush.9.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/bytownite_hs106.3b.depth.gz
./test/tetracorder/group.1.5um-broad/water-20C-10cm-path+chlrph-g.depth.gz
./test/tetracorder/group.1.5um-broad/organic_dry_long_grass-thick.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/water-20C-30cm-path-model.fd.gz
./test/tetracorder/group.1.5um-broad/water+mont50gpl+0.5veg.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/vegetation.dry+green.fd.gz
./test/tetracorder/group.1.5um-broad/water.turquoise-shallow.fd.gz
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_chlor+muscphy.fit.gz
./test/tetracorder/group.1.5um-broad/water-20C-100cm-path+chlrph-a.fit.gz
./test/tetracorder/group.1.5um-broad/water.high.chlorophyll2.fit.gz
./test/tetracorder/group.1.5um-broad/snow-ice-3mm+0.1water-273k.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/water+mont50gpl.fd.gz
./test/tetracorder/group.1.5um-broad/bytownite_hs106.3b.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/sphalerite_s26-341_1.5um.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/anorthite_hs201.3b.fit.gz
./test/tetracorder/group.1.5um-broad/water+mont5.01gpl.fd.gz
./test/tetracorder/group.1.5um-broad/snow.melting.1a.depth.gz
./test/tetracorder/group.1.5um-broad/water+mont100gpl.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/snow-ice-10mm+0.3water-273k.fit.gz
./test/tetracorder/group.1.5um-broad/snow.melting.8.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/olivine_fo18_ki3377.depth.gz
./test/tetracorder/group.1.5um-broad/snow.slush.9.fit.gz
./test/tetracorder/group.1.5um-broad/rutile_hs126.3b.depth.gz
./test/tetracorder/group.1.5um-broad/vegetation.dry+green.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/sphalerite_s26-341_1.5um.fit.gz
./test/tetracorder/group.1.5um-broad/olivine_fo89_gds70.a.fit.gz
./test/tetracorder/group.1.5um-broad/anorthite_hs201.3b.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/snow.melting.3.depth.gz
./test/tetracorder/group.1.5um-broad/snow.melting.1a.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/water+mont16.5gpl.depth.gz
./test/tetracorder/group.1.5um-broad/snow.melting.3.fit.gz
./test/tetracorder/group.1.5um-broad/water+mont5.01gpl.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/organic_dry_long_grass-thick.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/snow-ice-3mm+0.3water-273k.fd.gz
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_brd.br5a_actinolite.fit.gz
./test/tetracorder/group.1.5um-broad/rutile_hs126.3b.fit.gz
./test/tetracorder/group.1.5um-broad/water.low.chlorophyll-blue1.depth.gz
./test/tetracorder/group.1.5um-broad/snow-ice-3mm+0.3water-273k.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/organic_white_pvc_pipe.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_chlor+muscphy.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/olivine_fo91_gds71.b.fd.gz
./test/tetracorder/group.1.5um-broad/snow.slush.16.fit.gz
./test/tetracorder/group.1.5um-broad/olivine_fo89_gds70.d.fit.gz
./test/tetracorder/group.1.5um-broad/snow.and.ice-170k.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/water-20C-100cm-path+chlrph-g.fd.gz
./test/tetracorder/group.1.5um-broad/water.low.chlorophyll-blue1.fd.gz
./test/tetracorder/group.1.5um-broad/snow.slush.16.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/snow.slush.16.depth.gz
./test/tetracorder/group.1.5um-broad/snow.melting9+0.5veg.fd.gz
./test/tetracorder/group.1.5um-broad/water-20C-100cm-path+chlrph-a.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/olivine_fo66_ki3054.fd.gz
./test/tetracorder/group.1.5um-broad/water+mont100gpl+0.5veg.depth.gz
./test/tetracorder/group.1.5um-broad/sphalerite_s26-341_1.5um.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/snow.melting9+0.5veg.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/organic_green_plastic_tarp.fit.gz
./test/tetracorder/group.1.5um-broad/vegetation.dry+green.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/snow-ice-10mm+0.3water-273k.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/snow-ice-3mm-273k.depth.gz
./test/tetracorder/group.1.5um-broad/snow.and.ice-248k.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_brd.br5a_actinolite.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_brd.br5a_actinolite.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/water+mont16.5gpl.fd.gz
./test/tetracorder/group.1.5um-broad/water+mont100gpl.fd.gz
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_vbroad_br20.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/water-20C-30cm-path-model.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/olivine_fo18_ki3377.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/rutile_hs126.3b.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_nrw.cummingtonite.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_vbroad_br20.depth.gz
./test/tetracorder/group.1.5um-broad/snow.melting.8.fit.gz
./test/tetracorder/group.1.5um-broad/water.low.chlorophyll.fit.gz
./test/tetracorder/group.1.5um-broad/water.turquoise-shallow.fit.gz
./test/tetracorder/group.1.5um-broad/olivine_fo80_hs285.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/olivine_fo89_gds70.c.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/snow.melting9+0.5veg.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/water+mont1.67gpl.fd.gz
./test/tetracorder/group.1.5um-broad/water+mont5.01gpl.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/snow.melting16+0.5veg.fit.gz
./test/tetracorder/group.1.5um-broad/water+mont1.67gpl.fit.gz
./test/tetracorder/group.1.5um-broad/water.red.algae.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/water+mont1.67gpl.depth.gz
./test/tetracorder/group.1.5um-broad/water-20C-100cm-path+chlrph-g.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/snow-ice-3mm+0.2water-273k.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/olivine_fo66_ki3054.fit.gz
./test/tetracorder/group.1.5um-broad/water.high.chlorophyll.fd.gz
./test/tetracorder/group.1.5um-broad/water+mont0.5gpl.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/magnetite_hs195.3b.or.olivine.fd.gz
./test/tetracorder/group.1.5um-broad/water-20C-300cm-path+chlrph-g.fd.gz
./test/tetracorder/group.1.5um-broad/water+mont100gpl.fit.gz
./test/tetracorder/group.1.5um-broad/snow.melting.1a.fd.gz
./test/tetracorder/group.1.5um-broad/water.low.chlorophyll-blue1.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/olivine_fo18_ki3377.fit.gz
./test/tetracorder/group.1.5um-broad/olivine_fo60_ki3189.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/magnetite_hs195.3b.or.olivine.fit.gz
./test/tetracorder/group.1.5um-broad/olivine_fo18_ki3377.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/water+mont100gpl+0.5veg.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_chlorite.clinochlor.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/water+mont100gpl.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_nrw.cummingtonite.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/snow.slush.16.fd.gz
./test/tetracorder/group.1.5um-broad/water.high.chlorophyll.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/snow-ice-10mm+0.3water-273k.fd.gz
./test/tetracorder/group.1.5um-broad/organic_green_plastic_tarp.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/water+mont0.5gpl.fit.gz
./test/tetracorder/group.1.5um-broad/olivine_fo91_gds71.b.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/water+mont50gpl.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/snow.and.ice-248k.depth.gz
./test/tetracorder/group.1.5um-broad/organic_green_plastic_tarp.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/bytownite_hs106.3b.fd.gz
./test/tetracorder/group.1.5um-broad/olivine_fo80_hs285.fit.gz
./test/tetracorder/group.1.5um-broad/water.high.chlorophyll2.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/water-20C-300cm-path+chlrph-g.depth.gz
./test/tetracorder/group.1.5um-broad/vegetation1.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_nrw.cummingtonite.fit.gz
./test/tetracorder/group.1.5um-broad/olivine_fo80_hs285.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/water.red.algae.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/snow.slush.9.fd.gz
./test/tetracorder/group.1.5um-broad/water+mont1.67gpl.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/snow-ice-3mm+0.1water-273k.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/water.turquoise-shallow.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/snow.and.ice-170k.depth.gz
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_chlor+muscphy.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/vegetation.dry+green.fit.gz
./test/tetracorder/group.1.5um-broad/water+mont300gpl.fit.gz
./test/tetracorder/group.1.5um-broad/snow-ice-3mm+0.2water-273k.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/sphalerite_s26-341_1.5um.fd.gz
./test/tetracorder/group.1.5um-broad/snow.melting.8.depth.gz
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_chlor+muscphy.fd.gz
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_brd.br22c_actinolite.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/water+mont50gpl.fit.gz
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_chlor+muscphy.depth.gz
./test/tetracorder/group.1.5um-broad/bytownite_hs106.3b.fit.gz
./test/tetracorder/group.1.5um-broad/water.low.chlorophyll.fd.gz
./test/tetracorder/group.1.5um-broad/magnetite_hs195.3b.or.olivine.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/snow-ice-10mm-273k.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/water-20C-10cm-path-model.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/water-20C-100cm-path-model.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/water.turquoise-shallow.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/water.low.chlorophyll.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/snow-ice-10mm-273k.fit.gz
./test/tetracorder/group.1.5um-broad/water-20C-100cm-path-model.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/water.low.chlorophyll-turquoise.depth.gz
./test/tetracorder/group.1.5um-broad/water.red.algae.fd.gz
./test/tetracorder/group.1.5um-broad/olivine_fo89_gds70.d.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/snow-ice-3mm+0.3water-273k.depth.gz
./test/tetracorder/group.1.5um-broad/water+mont100gpl+0.5veg.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/water.red.algae.fit.gz
./test/tetracorder/group.1.5um-broad/water.high.chlorophyll.depth.gz
./test/tetracorder/group.1.5um-broad/organic_white_pvc_pipe.fd.gz
./test/tetracorder/group.1.5um-broad/olivine_fo89_gds70.a.fd.gz
./test/tetracorder/group.1.5um-broad/water-20C-100cm-path+chlrph-a.fd.gz
./test/tetracorder/group.1.5um-broad/snow-ice-10mm-273k.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_vbroad_br20.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/olivine_fo89_gds70.c.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/olivine_fo89_gds70.d.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/anorthite_hs201.3b.depth.gz
./test/tetracorder/group.1.5um-broad/water.turquoise-shallow.depth.gz
./test/tetracorder/group.1.5um-broad/snow.melting.1a.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_brd.br5a_actinolite.fd.gz
./test/tetracorder/group.1.5um-broad/olivine_fo91_gds71.b.depth.gz
./test/tetracorder/group.1.5um-broad/microcline_hs103.3b.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/organic_white_pvc_pipe.depth.gz
./test/tetracorder/group.1.5um-broad/rutile_hs126.3b.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/water-20C-100cm-path+chlrph-a.depth.gz
./test/tetracorder/group.1.5um-broad/snow.and.ice-170k.fd.gz
./test/tetracorder/group.1.5um-broad/water.turquoise-shallow.fd.gz.hdr
./test/tetracorder/group.1.5um-broad/olivine_fo89_gds70.d.fit.gz.hdr
./test/tetracorder/group.1.5um-broad/g4-fe2+generic_brd.br22c_actinolite.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/olivine_fo89_gds70.c.depth.gz
./test/tetracorder/group.1.5um-broad/water+mont300gpl.depth.gz.hdr
./test/tetracorder/group.1.5um-broad/water+mont0.5gpl.depth.gz

 

rimane il problema della visualizzazione  

mkdir -p ~/tetracorder_out
python tetracorder_iron_oxides.py \
    --group-dir group.2um \
    --filter "kaolgrp_|kaol|smectite_|micagrp_illite|micagrp_vermiculite|chlorite|palygorskite|sepiolite|saponite|hectorite|dickite|nacrite|halloysite|pyroph" \
    --output ~/tetracorder_out/argille_map.png \
    --geotiff ~/tetracorder_out/argille_map.tif \
    --title "Minerali argillosi" 

 

#!/usr/bin/env python3

"""

tetracorder_visualize.py

=========================

Visualizza una mappa di classificazione prodotta da USGS Tetracorder

(es. results.cluster, results.cluster.sorted, results.group.X, ...)

colorando ogni classe secondo il file color.keys prodotto dallo stesso run,

e generando una legenda con i nomi dei materiali.

USO

---

python tetracorder_visualize.py \

--cluster /path/results.cluster.sorted \

--colorkeys /path/color.keys \

--output mappa_materiali.png

Se il file .hdr (ENVI) ha lo stesso nome + ".hdr" o si chiama

"<cluster>.hdr", viene individuato automaticamente. In alternativa

si può passare esplicitamente con --hdr.

DIPENDENZE

----------

pip install numpy matplotlib

# opzionale ma consigliato (lettura ENVI con georeferenziazione):

pip install rasterio

NOTE SUL PARSING DI color.keys

-------------------------------

Il formato del file color.keys di Tetracorder non è uno standard fisso:

varia leggermente tra versioni. Questo script prova diversi pattern

comuni (vedi funzione `parse_color_keys`). Se il parsing fallisce o i

colori/nomi non tornano, lancia lo script con --debug-colorkeys per

vedere riga per riga cosa viene interpretato, e poi mandami un estratto

delle prime ~10 righe del tuo file color.keys così affino il parser.

"""

import argparse

import re

import sys

from pathlib import Path

import numpy as np

import matplotlib.pyplot as plt

from matplotlib.colors import ListedColormap, BoundaryNorm

from matplotlib.patches import Patch

# --------------------------------------------------------------------------

# Lettura immagine ENVI (results.cluster, results.group.X, ecc.)

# --------------------------------------------------------------------------

def find_hdr(cluster_path: Path) -> Path:

"""Cerca l'header ENVI associato al file dati."""

candidates = [

cluster_path.with_suffix(cluster_path.suffix + ".hdr"),

cluster_path.with_suffix(".hdr"),

Path(str(cluster_path) + ".hdr"),

]

for c in candidates:

if c.exists():

return c

raise FileNotFoundError(

f"Nessun file .hdr trovato per {cluster_path}. "

f"Passalo esplicitamente con --hdr."

)

def parse_envi_hdr(hdr_path: Path) -> dict:

"""Parser minimale di un header ENVI (testo chiave=valore)."""

text = hdr_path.read_text(errors="ignore")

info = {}

for key in ("samples", "lines", "bands", "data type",

"interleave", "byte order", "header offset"):

m = re.search(rf"{key}\s*=\s*(\S+)", text, re.IGNORECASE)

if m:

info[key.replace(" ", "_")] = m.group(1)

required = ("samples", "lines", "data_type")

missing = [k for k in required if k not in info]

if missing:

raise ValueError(f"Header ENVI incompleto, mancano: {missing}")

return info

ENVI_DTYPES = {

"1": np.uint8, "2": np.int16, "3": np.int32, "4": np.float32,

"5": np.float64, "6": np.complex64, "9": np.complex128,

"12": np.uint16, "13": np.uint32, "14": np.int64, "15": np.uint64,

}

def read_envi_raw(data_path: Path, hdr: dict) -> np.ndarray:

"""Legge un raster ENVI senza dipendenze esterne (fallback)."""

samples = int(hdr["samples"])

lines = int(hdr["lines"])

bands = int(hdr.get("bands", 1))

dtype = ENVI_DTYPES.get(hdr.get("data_type", "2"), np.int16)

offset = int(hdr.get("header_offset", 0))

byte_order = hdr.get("byte_order", "0")

interleave = hdr.get("interleave", "bsq").lower()

arr = np.fromfile(data_path, dtype=dtype, offset=offset)

expected = samples * lines * bands

arr = arr[:expected]

if byte_order == "1": # big-endian

arr = arr.byteswap().newbyteorder()

if interleave == "bsq":

arr = arr.reshape(bands, lines, samples)

elif interleave == "bil":

arr = arr.reshape(lines, bands, samples).transpose(1, 0, 2)

elif interleave == "bip":

arr = arr.reshape(lines, samples, bands).transpose(2, 0, 1)

else:

raise ValueError(f"Interleave ENVI non riconosciuto: {interleave}")

return arr[0] if bands == 1 else arr

def load_cluster_map(cluster_path: Path, hdr_path: Path | None) -> np.ndarray:

"""Carica la mappa raster, usando rasterio se disponibile."""

try:

import rasterio

with rasterio.open(cluster_path) as src:

return src.read(1)

except Exception:

pass # fallback manuale

if hdr_path is None:

hdr_path = find_hdr(cluster_path)

hdr = parse_envi_hdr(hdr_path)

band = read_envi_raw(cluster_path, hdr)

if band.ndim == 3:

band = band[0]

return band

# --------------------------------------------------------------------------

# Metadati di classificazione incorporati nell'header ENVI

# (class names / class lookup) - spesso è qui, non in un color.keys esterno

# --------------------------------------------------------------------------

def parse_envi_classification(hdr_path: Path) -> dict | None:

"""

Cerca nel .hdr i campi ENVI standard:

class names = {Unclassified, Kaolinite, Goethite, ...}

class lookup = {0,0,0, 255,0,0, 200,120,0, ...}

e ritorna {id: (r,g,b,nome)} se presenti, altrimenti None.

"""

text = hdr_path.read_text(errors="ignore")

def extract_braced(field):

m = re.search(rf"{field}\s*=\s*\{{([^}}]*)\}}", text,

re.IGNORECASE | re.DOTALL)

return m.group(1) if m else None

names_raw = extract_braced("class names")

lookup_raw = extract_braced("class lookup")

if not names_raw or not lookup_raw:

return None

names = [n.strip() for n in names_raw.split(",") if n.strip()]

nums = [int(x) for x in re.findall(r"-?\d+", lookup_raw)]

if len(nums) < len(names) * 3:

return None

result = {}

for i, name in enumerate(names):

r, g, b = nums[i * 3], nums[i * 3 + 1], nums[i * 3 + 2]

result[i] = (r, g, b, name)

return result

# --------------------------------------------------------------------------

# Parsing color.keys (file di testo esterno, se lo trovi separatamente)

# --------------------------------------------------------------------------

def parse_color_keys(path: Path, debug: bool = False) -> dict:

"""

Ritorna {id_materiale: (r, g, b, nome)}.

Prova più pattern, perché il formato varia tra run Tetracorder:

1) " 12 255 128 0 Kaolinite_CM9.spc"

2) "12 Kaolinite_CM9 R=255 G=128 B=0"

3. CSV-like: "12,255,128,0,Kaolinite"

"""

patterns = [

re.compile(r"^\s*(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\S.*)$"),

re.compile(r"^\s*(\d+)\s+(\S+).*?R\s*=?\s*(\d+).*?G\s*=?\s*(\d+).*?B\s*=?\s*(\d+)", re.IGNORECASE),

re.compile(r"^\s*(\d+)\s*,\s*(\d+)\s*,\s*(\d+)\s*,\s*(\d+)\s*,\s*(\S.*)$"),

]

result = {}

unmatched = []

for raw_line in path.read_text(errors="ignore").splitlines():

line = raw_line.strip()

if not line or line.startswith("#") or line.startswith(";"):

continue

m = patterns[0].match(line)

if m:

idx, r, g, b, name = m.groups()

result[int(idx)] = (int(r), int(g), int(b), name.strip())

continue

m = patterns[1].match(line)

if m:

idx, name, r, g, b = m.groups()

result[int(idx)] = (int(r), int(g), int(b), name.strip())

continue

m = patterns[2].match(line)

if m:

idx, r, g, b, name = m.groups()

result[int(idx)] = (int(r), int(g), int(b), name.strip())

continue

unmatched.append(raw_line)

if debug:

print(f"[debug] {len(result)} voci interpretate, "

f"{len(unmatched)} righe non riconosciute.")

for u in unmatched[:15]:

print(" ??", u)

if not result:

raise ValueError(

"Nessuna riga di color.keys è stata interpretata. "

"Rilancia con --debug-colorkeys e mandami le prime righe del file "

"cosi' adatto il parser al formato esatto."

)

return result

# --------------------------------------------------------------------------

# Plot

# --------------------------------------------------------------------------

def plot_classification(class_map: np.ndarray, color_keys: dict,

output: Path, title: str, max_legend_items: int = 40):

present_ids = sorted(i for i in np.unique(class_map) if i != 0 and i in color_keys)

if not present_ids:

print("Attenzione: nessun ID presente nella mappa ha una corrispondenza "

"in color.keys. Controlla che i due file provengano dallo stesso run.")

# id 0 = nessun match -> sfondo grigio chiaro

ids = [0] + present_ids

colors = [(0.92, 0.92, 0.92)]

labels = ["(nessun match)"]

for i in present_ids:

r, g, b, name = color_keys[i]

colors.append((r / 255, g / 255, b / 255))

labels.append(f"{i}: {name}")

cmap = ListedColormap(colors)

bounds = ids + [ids[-1] + 1]

norm = BoundaryNorm(bounds, cmap.N)

remap = np.zeros_like(class_map, dtype=np.int32)

for new_i, orig_i in enumerate(ids):

remap[class_map == orig_i] = new_i

fig, ax = plt.subplots(figsize=(10, 8))

ax.imshow(remap, cmap=ListedColormap(colors), interpolation="nearest")

ax.set_title(title)

ax.set_xticks([])

ax.set_yticks([])

legend_ids = present_ids[:max_legend_items]

handles = [Patch(facecolor=(color_keys[i][0] / 255,

color_keys[i][1] / 255,

color_keys[i][2] / 255),

label=f"{color_keys[i][3]}")

for i in legend_ids]

if len(present_ids) > max_legend_items:

handles.append(Patch(facecolor="white",

label=f"... +{len(present_ids) - max_legend_items} altri"))

ax.legend(handles=handles, bbox_to_anchor=(1.02, 1), loc="upper left",

fontsize=7, frameon=False, ncol=1)

fig.tight_layout()

fig.savefig(output, dpi=200, bbox_inches="tight")

print(f"Salvato: {output} ({len(present_ids)} materiali distinti trovati)")

# --------------------------------------------------------------------------

# Main

# --------------------------------------------------------------------------

def main():

p = argparse.ArgumentParser(description=__doc__,

formatter_class=argparse.RawDescriptionHelpFormatter)

p.add_argument("--cluster", required=True, type=Path,

help="File risultati (es. results.cluster.sorted)")

p.add_argument("--hdr", type=Path, default=None,

help="Header ENVI (.hdr), se non individuabile automaticamente")

p.add_argument("--colorkeys", required=False, type=Path, default=None,

help="File color.keys testuale (opzionale: se assente, "

"lo script cerca 'class names'/'class lookup' "

"direttamente nell'header ENVI del raster)")

p.add_argument("--output", type=Path, default=Path("tetracorder_map.png"),

help="File immagine di output (default: tetracorder_map.png)")

p.add_argument("--title", default="Classificazione Tetracorder")

p.add_argument("--debug-colorkeys", action="store_true",

help="Stampa il dettaglio del parsing di color.keys")

args = p.parse_args()

class_map = load_cluster_map(args.cluster, args.hdr)

color_keys = None

hdr_path = args.hdr or find_hdr(args.cluster)

if hdr_path.exists():

color_keys = parse_envi_classification(hdr_path)

if color_keys:

print(f"Trovate {len(color_keys)} classi nei metadati ENVI "

f"dell'header ({hdr_path.name}).")

if color_keys is None:

if args.colorkeys is None:

print("Nessuna classificazione trovata nell'header ENVI e "

"nessun --colorkeys fornito. Passa un file color.keys "

"testuale con --colorkeys, oppure mandami un estratto "

"del .hdr per capire dove sta l'informazione colore/nome.",

file=sys.stderr)

sys.exit(1)

color_keys = parse_color_keys(args.colorkeys, debug=args.debug_colorkeys)

plot_classification(class_map, color_keys, args.output, args.title)

if __name__ == "__main__":

sys.exit(main())

 

 

 

Elettronica strana

Ho trovato per terra questo piccolo circuito integrato...ha un pannello solare sul lato che non si vede, una batteria ricaricabile e un paio di led

Dovrebbe essere un fanale da bicicletta...quando lo si scuote si accendono i led

La cosa piu' curiosa e' le indicazioni dell'unico integrato sono state cancellate in fabbrica...immagino sia un microcontrollore

 

 

RTK data in DJI log

Rispetto al precedente post ho letto che il parser non estrae tutte le informazioni dal log.

A me interessano i dati RTK del drone. Per estrarre questi dati si deve primare estraree la keychain per individuare la posizione  

Cargo.toml 

[package]
name = "rtk_probe_local"
version = "0.1.0"
edition = "2021"

[dependencies]
dji-log-parser = "0.5"
serde_json = "1.0" 

 /src/main.rs

use dji_log_parser::record::Record;

use dji_log_parser::DJILog;

use std::env;

use std::fs;

fn main() {

let args: Vec<String> = env::args().collect();

if args.len() < 3 {

eprintln!("Uso: rtk_probe_local <path_al_txt> <api_key>");

std::process::exit(1);

}

let path = &args[1];

let api_key = &args[2];

let bytes = fs::read(path).expect("impossibile leggere il file");

let parser = DJILog::from_bytes(bytes).expect("impossibile leggere l'header");

println!("Versione log: {}", parser.version);

let keychains = if parser.version >= 13 {

Some(

parser

.fetch_keychains(api_key)

.expect("fetch_keychains fallita: controlla api_key/connessione"),

)

} else {

None

};

// Salva la keychain su disco per riuso futuro offline (evita nuove chiamate API)

if let Some(ref kc) = keychains {

let json = serde_json::to_string_pretty(kc).unwrap();

fs::write("keychains_cache.json", json).ok();

println!("Keychain salvata in keychains_cache.json (riusabile offline).");

}

let records = parser

.records(keychains)

.expect("errore nel parsing dei record");

println!("Totale record letti: {}", records.len());

let mut count_45 = 0usize;

let mut count_unknown_types: std::collections::HashMap<u8, usize> =

std::collections::HashMap::new();

for record in records.iter() {

if let Record::Unknown(record_type, data) = record {

*count_unknown_types.entry(*record_type).or_insert(0) += 1;

if *record_type == 45 {

count_45 += 1;

if count_45 <= 5 {

// stampa i primi 5 record RTK grezzi in hex

let hex: String = data.iter().map(|b| format!("{:02x}", b)).collect();

println!(

"[RTK record #{}] len={} bytes | hex: {}",

count_45,

data.len(),

hex

);

}

}

}

}

println!("\nRiepilogo tipi record 'Unknown' (non implementati dal crate):");

let mut types: Vec<_> = count_unknown_types.iter().collect();

types.sort();

for (t, c) in types {

println!(" tipo {:>3}: {} occorrenze", t, c);

}

println!(

"\nRecord tipo 45 (RTKDifferenceDataType) trovati: {}",

count_45

);

}

si creano i due file e si lancia

cargo build --release 

cargo run --release -- 'DJIFlightRecord_2026-07-31_[10-09-51].txt' XXXXXXXX8f100d529d2329caba3 keychain.json

a questo punto si puo' usare la keychain per estrarre i dati

python3 dji_rtk_extractor.py 'DJIFlightRecord_2026-07-31_[10-09-51].txt'  keychain.json -o output.csv 

 

dji_rtk_extractor.py 

#!/usr/bin/env python3

"""

dji_rtk_extractor.py

Estrae i dati RTK epoca-per-epoca (record binario tipo 45, "RTKDifferenceDataType")

dai flight log DJI in formato TXT (versione 13+, criptati AES-256-CBC), e li esporta

in un CSV con posizione rover (drone), posizione base station RTK, quota ellissoidica,

flag di qualita' del fix, e tempo di volo (fly_time) correlato dai record OSD adiacenti.

Il crate Rust ufficiale "dji-log-parser" non implementa questo tipo di record (lo

cattura come bytes grezzi non decodificati). Questo script reimplementa in Python

pura la pipeline di decodifica (XOR + AES-256-CBC) usata dal crate per ottenere il

contenuto in chiaro, poi interpreta la struttura byte-per-byte del record 45

(dedotta empiricamente e validata contro fly_time / geografia nota).

REQUISITI:

- una keychain JSON gia' ottenuta (vedi sotto), NON serve una API key DJI qui

dentro: la decrittazione AES e' un'operazione locale una volta che si hanno

le chiavi. La keychain si ottiene una sola volta per log, con una chiamata

all'API DJI (serve connettivita' verso i server DJI + una API key valida),

es. tramite il crate dji-log-parser stesso (DJILog::fetch_keychains) o via

il tool CLI ufficiale "dji-log". Una volta salvata come JSON, riusabile

offline per quello stesso file di log.

USO:

python3 dji_rtk_extractor.py FLIGHT_LOG.txt keychain.json -o output.csv

Se il log NON e' criptato (versione < 13), la keychain non serve:

python3 dji_rtk_extractor.py FLIGHT_LOG.txt -o output.csv

STRUTTURA DEL RECORD TIPO 45 (83 byte, dedotta e validata su un volo di test):

offset 0-10 : non identificato (11 byte, probabile satelliti/flag)

offset 11 : double -> latitudine rover (drone), gradi

offset 19 : double -> longitudine rover, gradi

offset 27 : float32 -> quota ellissoidica rover, metri

offset 31 : double -> latitudine base station RTK, gradi (costante nel volo)

offset 39 : double -> longitudine base station RTK, gradi (costante)

offset 47 : float32 -> quota ellissoidica base station, metri (costante)

offset 51 : float32 -> ipotesi: heading rover, gradi (0-360) - NON VALIDATO

offset 57 : float32 -> flag qualita' fix (1/2/3/4, verosimilmente

Fixed/Float/DGPS/Standalone)

offset 61-82 : non identificato (22 byte)

Questa struttura e' stata dedotta per un M400 con log versione 14; se noti valori

palesemente incoerenti (es. lat/lon fuori range) con un log di versione diversa,

gli offset potrebbero non essere validi 1:1 e vanno riverificati.

"""

import argparse

import base64

import csv

import json

import struct

import sys

from pathlib import Path

try:

from Crypto.Cipher import AES

from Crypto.Util.Padding import unpad

except ImportError:

sys.exit(

"Manca pycryptodome. Installa con:\n"

" pip install pycryptodome --break-system-packages"

)

# ----------------------------------------------------------------------------

# CRC64 (variante "Jones", usata da DJI per derivare la chiave XOR per-record)

# ----------------------------------------------------------------------------

_CRC64_JONES_POLY_REFLECTED = 0x95AC9329AC4BC9B5

def crc64(crc: int, data: bytes) -> int:

for byte in data:

crc ^= byte

for _ in range(8):

if crc & 1:

crc = (crc >> 1) ^ _CRC64_JONES_POLY_REFLECTED

else:

crc >>= 1

return crc

# ----------------------------------------------------------------------------

# XOR layer: ogni record e' prima offuscato con un keystream a 8 byte derivato

# dal primo byte del payload e dal record_type, poi (per i record non-plaintext)

# il risultato XOR e' il ciphertext AES-256-CBC vero e proprio.

# ----------------------------------------------------------------------------

_MAGIC = 0x123456789ABCDEF0

_MASK64 = (1 << 64) - 1

class XorKeystream:

def __init__(self, first_byte: int, record_type: int):

seed = (first_byte + record_type) & 0xFF

magic_mul = (_MAGIC * first_byte) & _MASK64

magic_bytes = magic_mul.to_bytes(8, "little")

key64 = crc64(seed, magic_bytes)

self.key = key64.to_bytes(8, "little")

def decode(self, data: bytes) -> bytes:

return bytes(b ^ self.key[i % 8] for i, b in enumerate(data))

# ----------------------------------------------------------------------------

# Mappatura record_type -> "feature point" (famiglia di chiavi nella keychain)

# ----------------------------------------------------------------------------

def feature_point(record_type: int, version: int) -> str:

def v13(base, other):

return base if version == 13 else other

version_dependent = {

3: ("BaseFeature", "GimbalFeature"),

4: ("BaseFeature", "RCFeature"),

7: ("BaseFeature", "BatteryFeature"),

8: ("BaseFeature", "BatteryFeature"),

11: ("BaseFeature", "RCFeature"),

22: ("AfterSalesFeature", "BatteryFeature"),

25: ("BaseFeature", "CameraFeature"),

28: ("AfterSalesFeature", "FlySafeFeature"),

29: ("BaseFeature", "RCFeature"),

33: ("BaseFeature", "RCFeature"),

51: ("AfterSalesFeature", "FlySafeFeature"),

52: ("AfterSalesFeature", "FlySafeFeature"),

53: ("AfterSalesFeature", "FlightHubFeature"),

}

fixed = {

1: "BaseFeature", 2: "BaseFeature", 5: "DJIFlyCustomFeature",

6: "BaseFeature", 9: "DJIFlyCustomFeature", 10: "DJIFlyCustomFeature",

12: "AfterSalesFeature", 13: "BaseFeature", 14: "BaseFeature",

15: "BaseFeature", 16: "AfterSalesFeature", 17: "VisionFeature",

18: "VisionFeature", 19: "AfterSalesFeature", 20: "DJIFlyCustomFeature",

21: "AgricultureFeature", 24: "DJIFlyCustomFeature", 26: "AfterSalesFeature",

27: "AfterSalesFeature", 30: "DJIFlyCustomFeature", 31: "WaypointFeature",

32: "WaypointFeature", 34: "WaypointFeature", 35: "WaypointFeature",

36: "WaypointFeature", 38: "WaypointFeature", 39: "WaypointFeature",

40: "BaseFeature", 41: "AgricultureFeature", 43: "AgricultureFeature",

44: "AgricultureFeature", 45: "AgricultureFeature", 46: "AgricultureFeature",

47: "AgricultureFeature", 48: "AgricultureFeature", 49: "AirLinkFeature",

50: "PlaintextFeature", 54: "DJIFlyCustomFeature", 55: "SecurityFeature",

56: "PlaintextFeature", 58: "BaseFeature", 59: "BaseFeature",

62: "RCFeature", 63: "BaseFeature",

}

if record_type in version_dependent:

base, other = version_dependent[record_type]

return v13(base, other)

return fixed.get(record_type, "PlaintextFeature")

_FEATURE_JSON_NAME = {

"BaseFeature": "FR_Standardization_Feature_Base_1",

"VisionFeature": "FR_Standardization_Feature_Vision_2",

"WaypointFeature": "FR_Standardization_Feature_Waypoint_3",

"AgricultureFeature": "FR_Standardization_Feature_Agriculture_4",

"AirLinkFeature": "FR_Standardization_Feature_AirLink_5",

"AfterSalesFeature": "FR_Standardization_Feature_AfterSales_6",

"DJIFlyCustomFeature": "FR_Standardization_Feature_DJIFlyCustom_7",

"PlaintextFeature": "FR_Standardization_Feature_Plaintext_8",

"FlightHubFeature": "FR_Standardization_Feature_FlightHub_9",

"GimbalFeature": "FR_Standardization_Feature_Gimbal_10",

"RCFeature": "FR_Standardization_Feature_RC_11",

"CameraFeature": "FR_Standardization_Feature_Camera_12",

"BatteryFeature": "FR_Standardization_Feature_Battery_13",

"FlySafeFeature": "FR_Standardization_Feature_FlySafe_14",

"SecurityFeature": "FR_Standardization_Feature_Security_15",

}

DJI_KEYCHAIN_ENDPOINT = "https://dev.dji.com/openapi/v1/flight-records/keychains"

_FEATURE_ID_TO_NAME = {

1: "BaseFeature", 2: "VisionFeature", 3: "WaypointFeature", 4: "AgricultureFeature",

5: "AirLinkFeature", 6: "AfterSalesFeature", 7: "DJIFlyCustomFeature", 8: "PlaintextFeature",

9: "FlightHubFeature", 10: "GimbalFeature", 11: "RCFeature", 12: "CameraFeature",

13: "BatteryFeature", 14: "FlySafeFeature", 15: "SecurityFeature",

}

def parse_auxiliary_metadata(raw: bytes):

"""Legge i blocchi Auxiliary::Info e Auxiliary::Version (a partire da offset 100,

valido per log versione >=12) per ottenere version/department richiesti dall'API DJI.

Ritorna (version_meta, department)."""

pos = 100 # PREFIX_SIZE

# Info block: magic(1) + length(2) + contenuto (length bytes, XOR'd, non ci serve)

magic0 = raw[pos]; pos += 1

if magic0 != 0:

raise ValueError(f"Atteso Auxiliary::Info (magic=0) a offset 100, trovato {magic0}")

len0 = struct.unpack_from("<H", raw, pos)[0]; pos += 2

pos += len0

# Version block: magic(1) + length(2) + version:u16 + department:u8 (+ padding)

magic1 = raw[pos]; pos += 1

if magic1 != 1:

raise ValueError(f"Atteso Auxiliary::Version (magic=1) a offset {pos-1}, trovato {magic1}")

len1 = struct.unpack_from("<H", raw, pos)[0]; pos += 2

version_meta = struct.unpack_from("<H", raw, pos)[0]

department = raw[pos + 2]

pos += len1

return version_meta, department, pos # pos = inizio atteso dei record (records_offset)

def extract_keystorage_entries(raw: bytes, records_offset: int, version: int):

"""Scansiona i record e ritorna [(feature_point_id, ciphertext_bytes), ...] dai

record tipo 56 (KeyStorage). Questi non sono AES-criptati (solo XOR), quindi non

serve nessuna keychain per leggerli: il loro contenuto E' l'input da mandare

all'API DJI per ottenere la keychain vera e propria."""

entries = []

for record_type, length_field, payload_start, payload in iter_records(raw, records_offset):

if record_type != 56:

continue

first_byte = payload[0]

xor = XorKeystream(first_byte, record_type)

content = xor.decode(payload[1:])

if len(content) < 4:

continue

feature_point_id = struct.unpack_from("<H", content, 0)[0]

data_length = struct.unpack_from("<H", content, 2)[0]

data = content[4:4 + data_length]

entries.append((feature_point_id, data))

return entries

def fetch_keychain_via_api(raw: bytes, records_offset: int, version: int, api_key: str,

endpoint: str = DJI_KEYCHAIN_ENDPOINT) -> dict:

"""Ricava automaticamente la keychain chiamando l'API DJI. Richiede connettivita'

di rete verso dev.dji.com e una API key valida. Ritorna un dict nello stesso

formato di load_keychain() (feature_name -> [iv_bytes, key_bytes]), e salva anche

la risposta raw in formato compatibile con keychains_cache.json per riuso offline."""

import urllib.request

import urllib.error

version_meta, department, _ = parse_auxiliary_metadata(raw)

entries = extract_keystorage_entries(raw, records_offset, version)

if not entries:

sys.exit("Nessun record KeyStorage trovato nel log: impossibile costruire la richiesta API.")

keychain_array = [

{

"featurePoint": _FEATURE_ID_TO_NAME.get(fid, "PlaintextFeature"),

"aesCiphertext": base64.b64encode(data).decode("ascii"),

}

for fid, data in entries

]

body = json.dumps({

"version": version_meta,

"department": department,

"keychainsArray": [keychain_array],

}).encode("utf-8")

req = urllib.request.Request(

endpoint,

data=body,

headers={"Content-Type": "application/json", "Api-Key": api_key},

method="POST",

)

print(f"Chiamata API DJI: {endpoint} (version={version_meta}, department={department}, "

f"{len(keychain_array)} feature points)")

try:

with urllib.request.urlopen(req, timeout=30) as resp:

response_json = json.loads(resp.read().decode("utf-8"))

except urllib.error.HTTPError as e:

body_text = e.read().decode("utf-8", errors="replace")

sys.exit(f"Errore HTTP {e.code} dall'API DJI: {body_text}")

except urllib.error.URLError as e:

sys.exit(

f"Impossibile raggiungere l'API DJI ({e}). Verifica la connettivita' di rete "

"verso dev.dji.com da questa macchina."

)

result = response_json.get("result", {})

if result.get("code", -1) != 0:

sys.exit(f"L'API DJI ha risposto con errore: {result.get('msg')}")

data = response_json.get("data")

if not data:

sys.exit("Risposta API DJI senza dati di keychain.")

# data ha la stessa forma di keychains_cache.json: List[List[{featurePoint,aesKey,aesIv}]]

cache_path = str(Path(sys.argv[1]).with_suffix("")) + "_keychain.json"

with open(cache_path, "w") as f:

json.dump(data, f, indent=2)

print(f"Keychain ottenuta e salvata in: {cache_path} (riusabile offline con l'argomento keychain_file)")

keychain = {}

for entry in data[0]:

fname = entry["featurePoint"]

key_bytes = base64.b64decode(entry["aesKey"])

iv_bytes = base64.b64decode(entry["aesIv"])

for internal, jsonname in _FEATURE_JSON_NAME.items():

if jsonname == fname:

keychain[internal] = [iv_bytes, key_bytes]

break

return keychain

def load_keychain(path: str) -> dict:

"""Carica keychains_cache.json -> dict feature_name -> [iv_bytes, key_bytes] (mutabile)."""

with open(path) as f:

keychain_json = json.load(f)[0] # usa il primo set di chiavi del file

keychain = {}

for entry in keychain_json:

fname = entry["featurePoint"]

key_bytes = base64.b64decode(entry["aesKey"])

iv_bytes = base64.b64decode(entry["aesIv"])

for internal, jsonname in _FEATURE_JSON_NAME.items():

if jsonname == fname:

keychain[internal] = [iv_bytes, key_bytes]

break

return keychain

# ----------------------------------------------------------------------------

# Parsing del file di log

# ----------------------------------------------------------------------------

def parse_header(raw: bytes):

"""Ritorna (detail_offset, detail_length, version)."""

detail_offset, detail_length, version, _unknown = struct.unpack_from("<QHBB", raw, 0)

return detail_offset, detail_length, version

def iter_records(raw: bytes, records_offset: int):

"""Itera (record_type, length_field, payload_start, payload_bytes) senza decrittare."""

pos = records_offset

end = len(raw)

while pos < end:

if pos + 1 > end:

return

record_type = raw[pos]

pos += 1

if pos + 2 > end:

return

length_field = struct.unpack_from("<H", raw, pos)[0]

pos += 2

if length_field <= 2:

return

payload_start = pos

if pos + length_field > end:

return

payload = raw[payload_start: payload_start + length_field]

pos += length_field

if pos >= end:

return

end_byte = raw[pos]

pos += 1

if end_byte != 0xFF:

# framing rotto: interrompe la scansione da qui in poi

return

yield record_type, length_field, payload_start, payload

def decrypt_record(record_type, length_field, payload, version, keychain):

"""Ritorna bytes in chiaro, oppure None se non decodificabile (plaintext o manca chiave)."""

fname = feature_point(record_type, version)

if fname == "PlaintextFeature" or fname not in keychain:

return None

first_byte = payload[0]

xor = XorKeystream(first_byte, record_type)

ciphertext_xored_input = payload[1:]

cipher_len = length_field - 2

if cipher_len <= 0 or cipher_len > len(ciphertext_xored_input):

return None

ciphertext = xor.decode(ciphertext_xored_input[:cipher_len])

if len(ciphertext) == 0 or len(ciphertext) % 16 != 0:

return None

iv, key = keychain[fname]

next_iv = ciphertext[-16:]

try:

cipher = AES.new(key, AES.MODE_CBC, iv)

plain_padded = cipher.decrypt(ciphertext)

try:

plain = unpad(plain_padded, 16)

except ValueError:

plain = plain_padded

except Exception:

keychain[fname][0] = next_iv

return None

keychain[fname][0] = next_iv # aggiorna IV "a catena" per il prossimo record della stessa feature

return plain

# ----------------------------------------------------------------------------

# Interpretazione del contenuto dei record

# ----------------------------------------------------------------------------

def parse_rtk_record(plain: bytes):

"""Interpreta un record tipo 45 (RTKDifferenceDataType) gia' decrittato."""

if len(plain) < 61:

return None

return {

"rover_lat": struct.unpack_from("<d", plain, 11)[0],

"rover_lon": struct.unpack_from("<d", plain, 19)[0],

"rover_ellip_height_m": struct.unpack_from("<f", plain, 27)[0],

"base_lat": struct.unpack_from("<d", plain, 31)[0],

"base_lon": struct.unpack_from("<d", plain, 39)[0],

"base_ellip_height_m": struct.unpack_from("<f", plain, 47)[0],

"heading_or_quality_deg": struct.unpack_from("<f", plain, 51)[0],

"fix_flag": struct.unpack_from("<f", plain, 57)[0],

}

def parse_osd_fly_time(plain: bytes):

"""Estrae fly_time (secondi) da un record OSD (tipo 1) gia' decrittato."""

if len(plain) < 44:

return None

fly_time_raw = struct.unpack_from("<H", plain, 42)[0]

return fly_time_raw / 10.0

# ----------------------------------------------------------------------------

# Pipeline principale

# ----------------------------------------------------------------------------

def extract_rtk(log_path: str, keychain_path: str | None, output_csv: str, api_key: str | None = None):

raw = Path(log_path).read_bytes()

detail_offset, detail_length, version = parse_header(raw)

print(f"Versione log: {version} | detail_offset: {detail_offset} | detail_length: {detail_length}")

records_offset = detail_offset if version > 12 else 100 # 100 = PREFIX_SIZE per log non criptati

if version >= 13 and not keychain_path and not api_key:

sys.exit(

f"Il log e' versione {version} (criptato). Serve una keychain JSON gia' salvata "

"(secondo argomento) oppure una API key DJI (--api-key) per recuperarla automaticamente."

)

if version >= 13 and api_key and not keychain_path:

keychain = fetch_keychain_via_api(raw, records_offset, version, api_key)

elif keychain_path:

keychain = load_keychain(keychain_path)

else:

keychain = {}

rtk_records = [] # (file_offset, plain_bytes)

osd_records = [] # (file_offset, fly_time_s)

n_total = 0

for record_type, length_field, payload_start, payload in iter_records(raw, records_offset):

n_total += 1

plain = decrypt_record(record_type, length_field, payload, version, keychain)

if plain is None:

continue

if record_type == 45:

rtk_records.append((payload_start, plain))

elif record_type == 1:

fly_time = parse_osd_fly_time(plain)

if fly_time is not None:

osd_records.append((payload_start, fly_time))

print(f"Record totali scansionati: {n_total}")

print(f"Record RTK (tipo 45) decodificati: {len(rtk_records)}")

print(f"Record OSD (tipo 1) decodificati: {len(osd_records)}")

if not rtk_records:

print("Nessun record RTK trovato: controlla che il log contenga davvero dati RTK, "

"o che la keychain sia quella corretta per questo file.")

osd_records.sort()

def nearest_fly_time(target_offset):

if not osd_records:

return None

best_off, best_ft, best_diff = None, None, None

for off, ft in osd_records:

diff = abs(off - target_offset)

if best_diff is None or diff < best_diff:

best_off, best_ft, best_diff = off, ft, diff

return best_ft

rows = []

for idx, (off, plain) in enumerate(rtk_records):

parsed = parse_rtk_record(plain)

if parsed is None:

continue

fly_time = nearest_fly_time(off)

rows.append({

"record_idx": idx,

"file_offset": off,

"fly_time_s": fly_time,

**parsed,

})

if rows:

fieldnames = list(rows[0].keys())

with open(output_csv, "w", newline="") as f:

w = csv.DictWriter(f, fieldnames=fieldnames)

w.writeheader()

w.writerows(rows)

print(f"\nScritte {len(rows)} epoche RTK in: {output_csv}")

else:

print("Nessuna epoca RTK esportata (0 righe).")

def main():

parser = argparse.ArgumentParser(

description="Estrae dati RTK epoca-per-epoca da un flight log DJI (TXT) in CSV.",

)

parser.add_argument("log_file", help="Percorso al file DJIFlightRecord_....txt")

parser.add_argument(

"keychain_file", nargs="?", default=None,

help="Percorso a keychains_cache.json gia' salvata (opzionale se si usa --api-key)",

)

parser.add_argument(

"-o", "--output", default="rtk_epochs.csv",

help="File CSV di output (default: rtk_epochs.csv)",

)

parser.add_argument(

"--api-key", default=None,

help="API key DJI: se fornita (e non si passa keychain_file), la keychain viene "

"recuperata automaticamente da dev.dji.com e salvata in <log>_keychain.json "

"per riuso futuro offline.",

)

args = parser.parse_args()

extract_rtk(args.log_file, args.keychain_file, args.output, api_key=args.api_key)

if __name__ == "__main__":

main()