lunedì 12 febbraio 2018

Realtime Hough Lines con OpenCV su Android

L'idea e' quella di utilizzare OpenCV per estrarre le linee in realtime dalle immagini riprese dalla fotocamera del telefono



Dopo avere impostato un nuovo progetto OpenCV su Android come visto qui ho modificato il secondo esempo (mixed_sample)

Un po' di commenti
1) il flusso di lavoro prevede di prendere l'immagine dalla fotocamera, convertirla in scala di grigi, applicare il filtro Canny, applicare l'algoritmo delle Hough Lines,  selezionare solo le linee maggiore di un valore limite (altrimenti l'algoritmo diventa rumoroso), calcolare il coefficiente angolare di ogni linea per creare 5 classi angolari (-90°/-60°,-60°/-30°,-30°/0°,0°/30°,30°/60°,60°/90°),  popolare una textbox in altro a sinistra con i valori delle classi,sovrapporre le Hough Lines all'immagine RGB (Img.procline) e mostrare il tutto sullo schermo

2) per poter modificare la textbox si deve procedere mediante RunonUIThread




uno dei problemini che si hanno usando i filtri con i parametri cosi' come sono impostati e' che
in scene reali complesse, il filtro canny puo' comportarsi in questo modo


rendendo totalmente inutile la ricerca delle houghlines. Per risolvere questo problema possono essere calcolati i valori di threshold in modo automatico dalla deviazione media dell'immagine come segue

 
MatOfDouble mu = new MatOfDouble();
MatOfDouble sigma = new MatOfDouble();

Core.meanStdDev(inputFrame.gray(),mu,sigma);

Mat lines = new Mat();
Imgproc.Canny(inputFrame.gray(), mIntermediateMat, mu.get(0,0)[0] - sigma.get(0,0)[0], mu.get(0,0)[0] - sigma.get(0,0)[0]); 

per fare le cose in modo piu' pulito possono essere usate le funzioni di OpenCV come il filtro adattativo

Imgproc.adaptiveThreshold(inputFrame.gray(),result,255,
Imgproc.ADAPTIVE_THRESH_GAUSSIAN_C,Imgproc.THRESH_OTSU,15,4)


Layout
---------------------------------
<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"    
xmlns:tools="http://schemas.android.com/tools"   
 xmlns:opencv="http://schemas.android.com/apk/res-auto"  
  android:layout_width="match_parent"    
android:layout_height="match_parent" >

    <org.opencv.android.JavaCameraView        
android:layout_width="fill_parent"        
android:layout_height="fill_parent"        
android:visibility="gone"        
android:id="@+id/tutorial1_activity_java_surface_view"        
opencv:show_fps="true"        
opencv:camera_id="any" />

    <LinearLayout        
android:layout_width="match_parent"        
android:layout_height="match_parent"        
android:orientation = "vertical" >

        <TextView            
android:layout_width="wrap_content"            
android:layout_height="wrap_content"            
android:id="@+id/fps_text_view"            
android:textSize="12sp"
            android:text="FPS:" />

    </LinearLayout>



</FrameLayout>

---------------------------------



Codice





---------------------------------







import android.app.Activity;
import android.graphics.Color;
import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.util.Log;
import android.view.Menu;
import android.view.MenuItem;
import android.view.SurfaceView;
import android.view.WindowManager;
import android.widget.TextView;

import org.opencv.android.OpenCVLoader;

import org.opencv.android.BaseLoaderCallback;
import org.opencv.android.CameraBridgeViewBase.CvCameraViewFrame;
import org.opencv.android.LoaderCallbackInterface;
import org.opencv.core.CvType;
import org.opencv.core.Mat;
import org.opencv.android.CameraBridgeViewBase;
import org.opencv.android.CameraBridgeViewBase.CvCameraViewListener2;
import org.opencv.core.Point;
import org.opencv.core.Scalar;
import org.opencv.imgproc.Imgproc;

import java.util.function.DoubleToLongFunction;

public class MainActivity extends Activity implements CvCameraViewListener2 {

    private static final String    TAG = "OCVSample::Activity";

    private static final int       VIEW_MODE_RGBA     = 0;
    private static final int       VIEW_MODE_GRAY     = 1;
    private static final int       VIEW_MODE_CANNY    = 2;
    private static final int       VIEW_MODE_FEATURES = 5;
    private static final int       VIEW_MODE_HOUGH = 3;


    private int                    mViewMode;
    private Mat                    mRgba;
    private Mat                    mIntermediateMat;
    private Mat                    mGray;

    private MenuItem               mItemPreviewRGBA;
    private MenuItem               mItemPreviewGray;
    private MenuItem               mItemPreviewCanny;
    private MenuItem               mItemPreviewFeatures;

    private CameraBridgeViewBase   mOpenCvCameraView;

    int [] classi = new int[10];


    int threshold = 50;
    int minLineSize = 20;
    int lineGap = 10;


    private BaseLoaderCallback  mLoaderCallback = new BaseLoaderCallback(this) {
        @Override        public void onManagerConnected(int status) {
            switch (status) {
                case LoaderCallbackInterface.SUCCESS:
                {
                    Log.i(TAG, "OpenCV loaded successfully");

                    mOpenCvCameraView.enableView();
                } break;
                default:
                {
                    super.onManagerConnected(status);
                } break;
            }
        }
    };
    private TextView testo;


    /** Called when the activity is first created. */    @Override    public void onCreate(Bundle savedInstanceState) {
        Log.i(TAG, "called onCreate");
        super.onCreate(savedInstanceState);
        getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);

        setContentView(R.layout.activity_main);

        testo = (TextView) findViewById(R.id.fps_text_view);
        testo.setTextColor(Color.RED);


        mOpenCvCameraView = (CameraBridgeViewBase) findViewById(R.id.tutorial1_activity_java_surface_view);
        mOpenCvCameraView.setVisibility(CameraBridgeViewBase.VISIBLE);
        mOpenCvCameraView.setCvCameraViewListener(this);
        mViewMode = VIEW_MODE_HOUGH;
    }

    @Override    public boolean onCreateOptionsMenu(Menu menu) {
        Log.i(TAG, "called onCreateOptionsMenu");
        mItemPreviewRGBA = menu.add("Preview RGBA");
        mItemPreviewGray = menu.add("Preview GRAY");
        mItemPreviewCanny = menu.add("Canny");
        mItemPreviewFeatures = menu.add("Find features");
        return true;
    }

    @Override    public void onPause()
    {
        super.onPause();
        if (mOpenCvCameraView != null)
            mOpenCvCameraView.disableView();
    }

    @Override    public void onResume()
    {
        super.onResume();
        if (!OpenCVLoader.initDebug()) {
            Log.d(TAG, "Internal OpenCV library not found. Using OpenCV Manager for initialization");
            OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_0_0, this, mLoaderCallback);
        } else {
            Log.d(TAG, "OpenCV library found inside package. Using it!");
            mLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);
        }
    }

    public void onDestroy() {
        super.onDestroy();
        if (mOpenCvCameraView != null)
            mOpenCvCameraView.disableView();
    }

    public void onCameraViewStarted(int width, int height) {
        mRgba = new Mat(height, width, CvType.CV_8UC4);
        mIntermediateMat = new Mat(height, width, CvType.CV_8UC4);
        mGray = new Mat(height, width, CvType.CV_8UC1);
    }

    public void onCameraViewStopped() {
        mRgba.release();
        mGray.release();
        mIntermediateMat.release();
    }

    public static boolean isBetween(double x, double lower, double upper) {
        return lower <= x && x <= upper;
    }


    public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
        final int viewMode = mViewMode;
        switch (viewMode) {
            case VIEW_MODE_GRAY:
                // input frame has gray scale format                


               Imgproc.cvtColor(inputFrame.gray(), mRgba, Imgproc.COLOR_GRAY2RGBA, 4);
                break;
            case VIEW_MODE_RGBA:
                // input frame has RBGA format                


               mRgba = inputFrame.rgba();
                break;
            case VIEW_MODE_CANNY:
                // input frame has gray scale format                mRgba = inputFrame.rgba();
                Imgproc.Canny(inputFrame.gray(), mIntermediateMat, 80, 100);
                Imgproc.cvtColor(mIntermediateMat, mRgba, Imgproc.COLOR_GRAY2RGBA, 4);


                break;
            case VIEW_MODE_HOUGH:

                mRgba = inputFrame.rgba();

                Mat lines = new Mat();
                Imgproc.Canny(inputFrame.gray(), mIntermediateMat, 80, 100);
                Imgproc.HoughLinesP(mIntermediateMat,lines, 1, Math.PI/180, threshold,minLineSize, lineGap);


                Log.d("Linee", Integer.toString(lines.rows()));



                for (int x = 0; x < lines.rows(); x++) {
                    double[] vec = lines.get(x, 0);
                    double x1 = vec[0],
                            y1 = vec[1],
                            x2 = vec[2],
                            y2 = vec[3];
                    Point start = new Point(x1, y1);
                    Point end = new Point(x2, y2);
                    double dx = x1 - x2;
                    double dy = y1 - y2;

                    double m = Math.toDegrees(Math.atan(dy / dx));
                    Log.d("Angolo", Double.toString(m));

                    double dist = Math.sqrt(dx * dx + dy * dy);
                    Log.d("Distanza", Double.toString(dist));


                    //calcola le classi in base all'angolo del coefficiente angolare
                    if (dist > 50.d)                    {
                                if (isBetween(m, -90.0, -60.0)) {
                                    classi[0] = classi[0] + 1;
                                }

                                if (isBetween(m, -60.0, -30.0)) {
                                    classi[1] = classi[1] + 1;
                                }


                                if (isBetween(m, -30.0, 0.0)) {
                                    classi[2] = classi[2] + 1;
                                }

                                if (isBetween(m, 0.0, 30.0)) {
                                    classi[3] = classi[3] + 1;
                                }

                                if (isBetween(m, 30.0, 60.0)) {
                                    classi[4] = classi[4] + 1;
                                }

                                if (isBetween(m, 60.0, 90.0)) {
                                    classi[5] = classi[5] + 1;
                                }


                        //sovrappone le Hough Lines
                        Imgproc.line(mRgba, start, end, new Scalar(255,0, 0, 1),2);
                }


                 

                }



                break;
            case VIEW_MODE_FEATURES:
                // input frame has RGBA format                mRgba = inputFrame.rgba();
                mGray = inputFrame.gray();
                break;
        }


        runOnUiThread(new Runnable() {
            @Override            public void run() {
                testo.setText(" -90/-60 "+Integer.toString(classi[0])+"\n" +
                              " -60/-30 "+Integer.toString(classi[1])+"\n" +
                              " -30/0   "+Integer.toString(classi[2])+"\n" +
                              "   0/30  "+Integer.toString(classi[3])+"\n" +
                              "  30/60  "+Integer.toString(classi[4])+"\n" +
                              "  60/90  "+Integer.toString(classi[5])+"\n"
                );
            }
        });

        return mRgba;
    }

    public boolean onOptionsItemSelected(MenuItem item) {
        Log.i(TAG, "called onOptionsItemSelected; selected item: " + item);

        if (item == mItemPreviewRGBA) {
            mViewMode = VIEW_MODE_RGBA;
        } else if (item == mItemPreviewGray) {
            mViewMode = VIEW_MODE_GRAY;
        } else if (item == mItemPreviewCanny) {
            mViewMode = VIEW_MODE_CANNY;
        } else if (item == mItemPreviewFeatures) {
            mViewMode = VIEW_MODE_FEATURES;
        }

        return true;
    }


}







---------------------------------
-
Etichette: , , ,

Nessun commento:

Posta un commento

Iscriviti a: Commenti sul post (Atom)

Update Plotly Dash Csv

    from dash import Dash , html , dcc , callback , Output , Input , State import plotly . express as px import pandas as pd import...

  • Creare un grafico animato con Python/Matplotlib
    In questo post viene indicato come creare uno scatterplot dinamico basato da dati ripresi da un file csv (nel dettaglio il file csv e' c...
  • Arduino e CH376S
    Questo post e' a seguito di quanto gia' visto nella precedente prova Lo scopo e' sempre il solito: creare un sistema che permet...
  • ESP32-2432S028R e LVGL
    La scheda ESP32-2432S028R monta un Esp Dev Module con uno schermo TFT a driver ILI9341 di 320x240 pixels 16 bit colore.Il sito di riferiment...