Feeds:
Posts
Comments

Archive for the ‘Android’ Category

I participated in a Lego Robot Competition recently, and our team of 5 persons won the first prize. Our robot successfully accomplished its mission: picking up a treasure, a Lego block in red color, and putting it in the dropping area inside a maze, exiting the maze, and hunting for the next treasure.

Components of the robot

robot001

Figure 1. The Lego Robot

The robot consists of a Samsung Galaxy S4 Android phone, a Rockwell MicroLogix 830 controller, 3 Lego motors, a Xiaomi power bank, and several infra-red distance sensors. The phone connects to the MicroLogix controller via an USB OTG cable, an USB to serial converter, and a serial cable. The communication protocol between the phone and the controller is MODBUS over serial line.

The Android app is the brain of the robot. It captures pictures periodically to detect treasures and the 3 mazes (in blue, green and purple color respectively) and reads the values of the sensors to determine what to do next. It sends commands to the controller to make a move, a turn, pick up or put down a treasure. The controller is not entirely passive: it can stop the robot when it detects that it is too close to a wall.

The PLC (Programmable Logical Controller) logic running in the controller uses PID to control the motors to make accurate moving or turning.

The Android app

The Android app is built on top of 2 open source libraries: OpenCV (http://opencv.org/) and the jamod Modbus library
(http://jamod.sourceforge.net/). The OpenCV library is used to capture pictures and detect objects from the pictures; while the jamod library is used to send commands and read sensor values to and from the MicroLogix controller. (I’ve made some modification to the jamod library to make it working with Android serial driver.)

At the heart of the Android app is the Robot class. A Robot object has a RobotCommander for sending commands to the MicroLogix controller, a RobotSensors object for reading sensor values, several ObjectDetectors for detecting treasures, mazes, outer walls, and the dropping area.

robot002

Figure 2. The class diagram of the Robot class.

State machines

The behavior of the Robot is controlled by the state machines. The top level state machine has 5 states: Initial, Initialized, Running, Paused, and Stopped. After the Robot is initialized it can be put into the Running state. When in the Running state, it can be Stopped, it can also be Paused and then resumed to the Running state.

robot003

Figure 3. The top level state machine

The Running state itself is a state machine that has 9 sub-states, as shown in Figure 4: FindingTreasure, ApproachingTreasure, PickingTreasure, FindingMaze, ApproachingMaze, FindingMazeEntrance, FindingDropArea, DroppingTreasure, and ExitingMaze.

The Robot first searches for a treasure. Once a treasure is found, it moves towards the treasure, and picks it up. It them searches for a maze, approaches it, and finds the entrance. Once it is in a maze, it looks for the drop area, drops the treasure, exits the maze, and starts over again.

robot004

Figure 4. The sub-state machine of the Running state

Each sub-state corresponds to a state machine, which we call a Sequence. Figure 5 shows the state diagram of the Finding Treasure state.

robot005

Figure 5. Finding Treasure sequence

The implementation of state machines

Two interfaces, IStateMachine and IState, as shown in Figure 6, are defined for implementing the state machines. IState represents a state. Upon entering or exiting a state, the enter() or exit() method is called respectively. The move() method will be called one or more times when the state is active to allow the Robot to make a move. IStateMachine represents a state machine. It can change the current active state and have the current state to make a move.

robot006

Figure 6. IStateMachine and IState

Figure 7 shows all classes that implement the IStateMachine interface. Class RobotStateMachine implements the top-level state machine as shown in Figure 3 and class RobotRunningStateMachine implements the sub-state machine as shown in Figure 4. Class RobotRunningSequence is the base class of sequences. Each sub-class of RobotRunningSequence corresponds to a state of the sub-state machine.

robot007

Figure 7. State machines

When the Robot object is in the Running state, the continueMission() method is called periodically, which calls move() on the RobotStateMachine object, which calls move() on the current state, Running, which calls move() on the RobutRunningStateMachine object, which calls move() on the current sequence. If the m_nextState member variable is not null, it calls changeState(), which calls exit() on the current state, makes m_nextState the current state, and calls enter() on it. It finally calls move() on the current state, which would most likely calls sendCommand on the Robot object to send a command to the controller.

robot008

Figure 8. State machines sequence diagram

The states of a state machine are implemented using a Java enum, with each enum field representing a state. The following code shows how the top-level states are defined.

public enum RobotState implements IState {
    Initial {
        @Override
        public void enter(IStateMachine machine) {
        }
        @Override
        public void move(IStateMachine machine) {
            machine.robot().initialize();
            machine.changeState(Initialized);
        }
        @Override
        public void exit(IStateMachine machine) {
        }
     },
     Initialized {
        @Override
        public void enter(IStateMachine machine) {
            machine.robot().ensureArmAtHome();
        }
        @Override
        public void move(IStateMachine machine) {
            // wait
        }
        @Override
        public void exit(IStateMachine machine) {
        }
 
    },
    Running {
        @Override
        public void enter(IStateMachine machine) {
            m_runningMachine.afterResume();
            machine.robot().log("Enter Running state", 2);
        }
        @Override
        public void move(IStateMachine machine) {
            m_runningMachine.move();
        }
        @Override
        public void exit(IStateMachine machine) {
            // do nothing
        }
 
    }, 
    Paused {<
        @Override
        public void enter(IStateMachine machine) {
            m_runningMachine.beforePause();
        }
        @Override
        public void move(IStateMachine machine) {
            machine.waitForResume();
        }
        @Override
        public void exit(IStateMachine machine) {
        }
 
    },
    Stopped {
        @Override
        public void enter(IStateMachine machine) {
            machine.robot().sendCommand(RobotAction.Stop);
        }
        @Override
        public void move(IStateMachine machine) {
            machine.robot().sleep(200);
        }
        @Override
        public void exit(IStateMachine machine) {
        }
    }
}

ThreadingEvery enum field is in fact a class. This makes the code more concise by not having to explicitly define a Java class for each state.

In an Android app, if the UI thread is blocked for more than 5 seconds the app will be forced to be stopped. To avoid running into this scenario, the entire state machine is running on a worker thread. When a Robot object is created, a work thread is created with a Runnable object whose run() method periodically calls the continueMission() method, which calls the move() method on the RobotStateMachine object.

In the TestSequenceActivity, a RobotTask class, inheriting from AsyncTask, is defined to test individual sequences using a worker thread.

In order to avoid the changeState() and move() methods of RobotStateMachine to be called by the UI thread, the methods throw an exception if the calling thread is the main thread.

public void move() {
    if (Looper.myLooper() == Looper.getMainLooper()) {
    throw new RuntimeException("move() cannot be called by the main thread");
}
…
}

Debugging

When the Android app is running, we cannot use Eclipse to debug the app as the USB port is used to connect to the controller. We chose to use log to help us understand what is going on. We create some Android Activities specially for debugging, e.g. an Activity for testing Modbus communication, an Activity for testing sending commands to the controller, and an Activity for testing individual sequences. The log of the debugging Activities provides much detailed information, while the log of the run mode Activity, RobotActivity, provides only important information.

An interface named IRobotLogger is defined for the Activities to implement to display logging information. When the log() method is called, if the logging level is equal to or higher than the lowest  logging level it is interested in, it appends the string passed in to the TextView.

Since the state machines are all running on a worker thread, the log() method cannot directly call the append() method on the TextView, instead it should send a request to the UI thread to do so by calling the runOnUiThread() on the Activity.

public void log(final String msg) {
    TestSequenceActivity.this.runOnUiThread(new Runnable() {
        public void run() {
            TextView logDisplay = (TextView) findViewById(R.id.text_log);
            logDisplay.append(msg + "\n");
        }
    });
}

Points of Interest

Adopting the State design pattern makes our code much easier to understand and debug. Even though only myself was working on the Java code debugging, the well understood behavior of each state machine plus the detailed log allow other team member to help me understand what was going on, and thus greatly shorten our debugging time.

When we designed the classes of the model, we had unit-testability in mind, although we hadn’t written much unit tests, as we were implementing the App in a rush. We defined some interfaces, such as IRobotCommander, IRobotSensors, to make them mockable.

Debuggability is the key for us to develop the app in such a short time with relatively good quality. The activities developed specially for debugging help us troubleshoot the issues we encountered quickly.

Using the Modbus protocol for the communication between the phone and the controller removes a lot of burden in both the Android app and the PLC code compare with using serial port communication directly.

If I had a chance to develop a similar app, I would consider developing a DSL (Domain Specific Language) for defining the sequences: defining how the states should be changed depending on the sensor values and the object detection result. This would allow non-Java developers to define and debug the sequences. I would also enhance the object detection by utilizing more OpenCV functionality.

 

 

Advertisements

Read Full Post »

It took my quite some time to install the latest PhoneGap (2.9.1 as for now) for Android on Windows 8. I’ve found this Blog particularly useful.

If your target Android version is 4.4.2 (API 19), make sure you have installed “Android SDK Build-tools 19.1”. Otherwise you may get an error something like

The SDK Build Tools revision (19.0.3) is too low for project 'HellowPhoneGap'.
Minimum required is 19.1.0

 

Read Full Post »

MathGrapher (Android app)

I’ve published an Android app, MathGrapher, a while ago. It seems to be attracting more and more users.

With MathGrapher, you can visualize function equations, parametric equations, and/or polar equations. You can also animate an equation by defining the range of one or more parameters: a, b, c, or d.

Here are some examples:

Function equation: y = a*x^2 + b*x + c
Parametric equation: x = a*sin(t); y = b*cos(t)
Polar equation: r = a*cos(b*t + pi/2)

You can select equations from the equation gallery that contains some predefined equations.

You can save the equations for later use.

Read Full Post »

I’ve published a new Android app, SATWordPuzzle, with which you can play CrossWord and WordSearch puzzle games as well as learn SAT vocabulary.

SATWordPuzzle generates CrossWord and WordSearch puzzles from over 5000 SAT vocabulary. You can also learn the definitions of the words, and add them as favorites. The favorite words have more chance to appear in the puzzles.

When you solve a CrossWord puzzle, you can swipe you finger on a word to show the clue of the word, or you can touch the Clue button to show the clues of all words.

You can turn on Instant Feedback to allow the app to change the background color of a letter right after you input one in a cell.

Read Full Post »

In some of my apps, I need to use a pre-existing database, which is very likely to be upgraded in a later release and which could be pretty big (over the roughly 1MB limit of an asset file). After some experiments and searching on the Web (especially on StackOverflow), I’ve settled with the following code. The basic idea is

1. if the database doesn’t exist yet when it is to be opened, combine the split files into a database file, and open it.

2. if the database version is changed, delete the database file in onUpgrade().

//The Android's default system path of your application database.
private static final String DB_PATH = "/data/data/com.mypackage.myapp/databases/";
private static final String DB_NAME = "mydb.db";
private static final int DB_VERSION = 2;
private static final String DB_SPLIT_NAME = "mydb.db.00";
private static final int DB_SPLIT_COUNT = 3;
private SQLiteDatabase m_database;
private final Context m_context;

/**
 * Constructor
 * Takes and keeps a reference of the passed context in order to access to the application assets and resources.
 * @param context
 */
public MyDB(Context context) {
 super(context, DB_NAME, null, DB_VERSION);
 this.m_context = context;
}

public static MyDB openDatabaseReadOnly(Context context) {
 MyDB db = new MyDB(context);

 try {
   db.createDataBase();
 } catch (IOException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
 }

 db.openDataBase(SQLiteDatabase.OPEN_READONLY);
 return db;
}

public static MyDB openDatabaseReadWrite(Context context) {
 MyDB db = new MyDB(context);

 try {
   db.createDataBase();
 } catch (IOException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
 }

 db.openDataBase(SQLiteDatabase.OPEN_READWRITE);
 return db;
}

/**
 * Creates an empty database on the system and rewrites it with your own database.
 */
public void createDataBase() throws IOException {
  boolean dbExist = checkDataBase();
  if (dbExist) {
    /* By calling this method here onUpgrade() will be called on a
    ** writable database, but only if the version number has been bumped.
    */
    SQLiteDatabase db = this.getWritableDatabase();

    if (db != null) {
      db.close();
    }
 }

 dbExist = checkDataBase();

 if (!dbExist) {
   try {
     /* By calling this method an empty database will be created into the 
     * default system path of your application so we are gonna be able 
     * to overwrite that database with our database.
     */
     SQLiteDatabase db = this.getReadableDatabase();

     if (db != null) {
       db.close();
     }
     copyDataBase();
   }
   catch (IOException e) {
     Log.e("DB", e.getMessage());
      throw new Error("Error copying database");
    }
  }
}

/**
 * Check if the database already exist to avoid re-copying the file each time you open the application.
 * @return true if it exists, false if it doesn't
 */
private static boolean checkDataBase(){
   SQLiteDatabase checkDB = null;
   try {
     String path = DB_PATH + DB_NAME;
     checkDB = SQLiteDatabase.openDatabase(path, null, SQLiteDatabase.OPEN_READONLY);
   }
   catch (SQLiteException e){
     //database does't exist yet.
   }

   if (checkDB != null) {
     checkDB.close();
   }

   return checkDB != null ? true : false;
}

/**
 * Copies your database from your local assets-folder to the just created empty database in the
 * system folder, from where it can be accessed and handled.
 * This is done by transferring byte stream.
 * */
private void copyDataBase() throws IOException {
  // Path to the just created empty db
  String outFileName = DB_PATH + DB_NAME;
  //Open the empty db as the output stream
  OutputStream output = new FileOutputStream(outFileName);
  //transfer bytes from the inputfile to the outputfile
  byte[] buffer = new byte[1024*8];

  AssetManager assetMgr = m_context.getAssets();

  for (int i = 1; i &lt;= DB_SPLIT_COUNT; i++) {
     //Open your local split file as the input stream
     String fn = DB_SPLIT_NAME + String.valueOf(i);
     InputStream input = assetMgr.open(fn);
     //Log.i("DB", "opened " + fn);

     int length;
     while ((length = input.read(buffer)) &gt; 0) {
       //Log.i("DB", "read " + String.valueOf(length));
       output.write(buffer, 0, length);
       //Log.i("DB", "write " + String.valueOf(length));
     }
     input.close();
   }

   //Close the streams
   output.flush();
   output.close();
}

private void openDataBase(int flags) throws SQLException{
  //Open the database
  String myPath = DB_PATH + DB_NAME;
  m_database = SQLiteDatabase.openDatabase(myPath, null, flags);
}

@Override
public synchronized void close() {
  if (m_database != null)
    m_database.close();
    super.close();
  }
}

@Override
public void onCreate(SQLiteDatabase db) {
  // do nothing
}

@Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
  if (newVersion &gt; oldVersion) {
     m_context.deleteDatabase(DB_NAME);
  }
}

Read Full Post »

有用户反映成語縱橫1.3版似乎变慢了。在1.3版我主要引入了即时反馈,即在一个成语所有的字被找出后,马上自动改变成语的背景色。另一个改动是利用Android的GestureDector检测单点触击和滑动手指。这两项改动都可能影响速度。

在下一版,我将在设置中增加一个选项,允许用户选择是否提供即时反馈。同时改进GestureDector的用法。

在新版推出之前,请尽量通过滑动手指一次选择多个字。这样会大大提高游戏的流畅性。这也同样适合于古诗纵横游戏。

Read Full Post »

刚刚发布了古诗纵横(繁体版和简体版)。

古诗纵横采用与成语纵横相同的填字游戏生成引擎,从106首脍炙人口的古诗中随机生成“找诗句”或“填字”游戏。在游戏过程中,可参考提示。

在玩找诗句游戏时,可单选某一个字,或通过滑动手指一次选择某诗句的所有的字。在一个诗句的所有的字都被选出后,该诗句会自动变色,表示该诗句已被找出。

在玩填字游戏时,沿着某一个诗句的方向滑动手指,能自动调出该诗句的提示。

你可以读任意一首古诗、或某作者的诗。你还可以查找诗句。每首诗的关键字都有注释。

Read Full Post »

Older Posts »