Predicting Ethnicity based on Person Name using Machine Learning

Recently, I stumbled upon a Python app called ethnicolr that uses deep learning to predict ethnicity based on a person forename and surname.  The app uses LSTM on sequences of bigrams generated from surname and forename as features.  Three different models were trained using data collected from US Census, Florida voter registration and Wikipedia.  Of the three, the wikipedia dataset appears the most interesting because of the broader categories of ethnicity available than others.  The categories are East Asian, Japanese, Indian SubContinent, Africans, Muslim, British, East European, Jewish, French, Germanic, Hispanic, Italian, and Nordic.  The reported precision and recall of the LSTM model using the wikipedia dataset are both approximately 73%.

Intrigued by the approach, I was curious how the performance would fare by using more traditional classifiers such as logistic regression which tends to take less time to train than LSTM.  The full Jupyter notebook is available here.

In summary, logistic regression produces slightly better precision and recall than LTSM and also took much less time to train (a few minutes versus about an hour on a 5 year old computer).  At the time of writing, average precision is 75% and recall is 76%.  The full confusion matrix is:

Perhaps as expected, some races such as Africans and Jewish are harder to predict than others due to more mixing and displacements over generations.

The major differences between the LSTM and regression model is that the former uses only bigrams while the latter uses bigrams, trigrams and quadgrams as features.  The regression model also weighs the features by applying tf-idf.  In addition, the regression model includes the middle name whenever it’s available though the performance improvement from adding middle name was observed to be negligible.

As a quick demo, the regression model was run on the following world leaders to predict their ethnicity:

Name	Ethnicity
Abe Shinzo	Japanese
Conte Giuseppe	Italian
Johnson Boris	British
Modi Narendra	Indan SubContinent
Netanyahu Benjamin	Jewish
Obama Barrack	Africans
Putin Vladimir	East European
Xi Jinping	East Asian

It's worthy to note that a few other approaches were attempted to improve precision and recall such as applying XGBoost and dataset balancing techniques.  At best, their performance was on par with logistic regression.  It is, of course, conceivable that more tuning might have been required for those methods to excel.

Halloween Edition: Dilbert Pointy-haired Boss with Arduino

What’s the best way to celebrate Halloween in the office? How about making the pointy-haired boss character from Dilbert that lip syncs to sound using nothing more than a cereal box and a micro-controller? Imagine the laughters you will get when you bring it to a meeting, put it on the table and have it start lip syncing to the conversation in the room.  Your boss may also appreciate your sense of humor (or not)!

Here is a video of the pointy-haired boss lip syncing to sound played on my computer.

To make the pointy-haired boss, all you need is an hour of art and craft, and the following inexpensive items:

• Cereal box
• Corrugated cardboard
• Color marker (black and orange)
• Paperclip
• Double sided tape
• Paper glue
• Printer papers (3 sheets)
• Arduino UNO R3
• Arduino jumper wires (6 pcs)
• USB cable (for Arduino)
• Tower Pro Servo and gear
• KY-037 sound sensor
• 220 Ohm resistor (optional)

Instead of Ardunio UNO R3 as the micro-controller, you can also get by with cheaper models from Arduino or clones.

How to Prepare the Cereal Box?

You can probably figure it out from watching the video above but here are the steps:

1. Cover up the cereal box with printer papers using glue
2. Cut the eyes, eyebrows, hair and lips from corrugated cardboard
3. Paint the hair pieces with black marker
4. Paint the eyes with orange marker
5. Tape or glue the facial features to the cereal box
6. Cut a small round nose hole for the microphone
7. Cut open a sizable mouth flap

Here are a few more mugshots to help you see the design from different angles.

How to Mount the Microphone as Nose?

Build a small triangular bracket from corrugated cardboard box.



Similar to how you would use a shelf bracket, tape the sound sensor onto the bracket, then tape the bracket inside of the cereal box wall where the nose is at.  The sensor microphone should protrude a little bit out of the nose hole.

How to Connect the Mouth Flap to Servo?

Mount the plastic gear to the servo so that it's parallel to the servo.  Bend the paper clip into shape and hook one end to the servo gear.  

Cut two small rectangles from corrugated cardboard and glue them to the back of the mouth flap.  The rectangles serve as a hinge for the mouth flap.

Insert the other end of the paperclip into the corrugated rectangle. The pictures below show the paperclip partially inserted but once fully inserted, the paperclip is locked in place between the two rectangles preventing it from sliding out.

Tape the servo inside the cereal box on the bottom at a location that lifts up the mouth flap.  In other words, the mouth is closed at this servo placement.

For my cereal box, this location is near the center.

How to Wire up the Circuit?

Using the jumper cables,

1. Connect the sound sensor analog input to pin A0 of Arduino
2. Connect the sound sensor power to +5 and GND of Arduino
3. Connect the servo control to pin 9 of Arduino
4. Connect the servo power to +3.3 and GND of Arduino

As best practice, it's recommended to bridge a 220 Ohm resistor between the servo and Arduino instead of connecting the two directly.

How to Program the Arduino?

Of all the steps, this is the easiest.  Upload the following "sketch" using the Arduino IDE.

#define MAXELMS 5

#include <Servo.h>

int inputs[MAXELMS];
bool isInputIncreasing;
Servo servo;
int servoPin = 9;
int sensorPin = A0;
long lastTime = millis();
int angleClose = 0;
float angleOpen = 20;

void setup() {
  servo.attach(servoPin);
  pinMode(sensorPin, INPUT);
  servo.write(angleClose);
}
  
void loop() {
  // store MAXELMS number of samples
  for (int i=1; i<MAXELMS; i++) {
    inputs[i-1] = inputs[i];
  }
  inputs[MAXELMS-1] = analogRead(sensorPin);

  // compute if input values are increasing
  isInputIncreasing = true;
  for (int i=1; i<MAXELMS; i++) {
    isInputIncreasing &= inputs[i] >= inputs[i-1];
  }

  long elapsedTime = millis() - lastTime;

  if (isInputIncreasing and elapsedTime > 120) {

    servo.write(angleOpen);
    delay(180);  
    servo.write(angleClose); 
    lastTime = millis();
  }
}

The analog input of the KY-037 sound sensor samples sound waves as unsigned integers.  The code looks for a rising wave signal as trigger.  When detected, it instructs the servo to rotate 20 degree and holds for 180 milliseconds before returning to 0 degree.  At 0 degree, the mouth is closed.  At 20 degree, the mouth is open.

Troubleshooting Tips

• If the mouth starts moving continuously without any apparent external sound trigger, it may be due to feedback noise from the movement of the mouth itself.  Try cushioning the sound sensor from the cereal box with something soft like a piece of sticky chewing gum.

• If the servo jams, try increasing the two delay timers in the code.

Happy Halloween!

Extract, Transform and Load (ETL) in Java

Extract, Transform and Load (ETL) is a common operation in data processing.  A recent ETL task of mine involves cleansing CSV files which contain millions of records.  The cleansing is done by calling an RESTful API which takes a batch of at most 100 records at a time, processes them and returns the results.  The API is accessible via a HTTP request and up to 4 concurrent requests are allowed.  Naturally, to process the CSV file quicker, one should take advantage of the concurrency allowed by the API.  While a relatively straightforward ETL task, a slight complication is that the machine where the Java program is scheduled to run regularly has a memory limitation.  It’s not possible to load the entire file into memory at any time during the execution of the Java program.  To work around the limitation, the CSV file would need to be loaded into memory a portion at a time.

There are many ways to implement the task in Java.  The old way, before Java 8, would be to first split the input CSV file into 4 separate files and then spawn a thread for each separate file.  Each thread would read 100 records at a time into memory from the assigned file, pass the records to the API and write the results to a temporary output file.  When the threads are finished, combine the temporary output files together into a single output file.  This way works just fine and it’s sufficient for the task at hand but what if you were to use Stream and functional programming in Java 8?

The easiest (but not the most elegant) approach without loading the entire CSV file into memory is to first read the file and divide it into batches of 100 records by recording the line number of the first record of each batch into a list.  For example, an input CSV file with 333 rows would produce line number 2, 102, 202 and 302 in the list.  The first batch starts at line number 2, the second batch at line number 102 and so forth.  The code is:

 public static List<Integer> partition(Path filePath, int batchSize) throws IOException {
     List<Integer> lineNumbers = new ArrayList<>();
     LineNumberReader rdr = new LineNumberReader(Files.newBufferedReader(filePath));
     rdr.readLine(); // skip header row
     while (rdr.readLine() != null) {
         int lineNumber = rdr.getLineNumber();
         if ((lineNumber - 2) % batchSize == 0) {
             lineNumbers.add(lineNumber);
         }
     }
     rdr.close();
     return lineNumbers;
 }

Next, we create a parallel stream from the list of line numbers by calling:

 Stream<Integer> stream = lineNumbers.stream().parallel();

By default, the number of parallel streams is the same as the number of processors.  The stream is then piped to the extract method:

 Stream<List<String>> data = stream.map(i -> CsvBatchProcessor.extract(filePath, i, batchSize));

The extract method returns the next 100 records from the CSV file starting from the given line number:

 public static List<String> extract(Path path, int startLineNumber, int batchSize) {
     try {
         // load a batch of data rows from file to memory
         List<String> rows = new ArrayList<>(); 
         LineNumberReader rdr = new LineNumberReader(Files.newBufferedReader(path));
         for (String line = null; (line = rdr.readLine()) != null;) {
             int lineNumber = rdr.getLineNumber();
             if (lineNumber >= startLineNumber && lineNumber < startLineNumber + batchSize) {
                 rows.add(line);
             }   
         }   
            
         return rows;
     } catch (IOException e) {
         throw new RuntimeException(e);
     }   
 }

The next step is to process the 100 records by calling:

 data = data.map(CsvBatchProcessor::transform);

The transform method is where the API is called:

 public static List<String> transform(List<String> rows) {
     List<String> newRows = new ArrayList<>();
     for (String row : rows) { 
         newRows.add(row.toUpperCase());
     }
        
     return newRows; 
 }

Finally, we output the processed data to the console so it can redirected to a file.

 data.flatMap(x -> x.stream()).forEach(System.out::println);

The careful reader may have noticed that the output CSV may not retain the same ordering as the input CSV because of the use of parallel stream.  This is definitely the case since there’s no guarantee to the ordering of the stream threads execution.  For my requirements, the ordering is not important so I’ve neglected it.

The full source code is:

 import java.util.List;
 import java.util.ArrayList;
 import java.util.stream.Stream;
 import java.io.LineNumberReader;
 import java.io.IOException;
 import java.nio.file.Files;
 import java.nio.file.Paths;
 import java.nio.file.Path;

 /**
  * Perform ETL on an input file by a batch of rows at a time to conserve computational resources.
  */
 public class CsvBatchProcessor {

    /**
     * Get the starting line number for each batch.
     */
    public static List partition(Path filePath, int batchSize) throws IOException {
        List lineNumbers = new ArrayList<>();
        LineNumberReader rdr = new LineNumberReader(Files.newBufferedReader(filePath));
        rdr.readLine(); // skip header row
        while (rdr.readLine() != null) {
            int lineNumber = rdr.getLineNumber();
            if ((lineNumber - 2) % batchSize == 0) {
                lineNumbers.add(lineNumber);
            }
        }
        rdr.close();

        return lineNumbers;
    }
    
    /**
     * Extract a batch of rows from data file.
     */
    public static List extract(Path path, int startLineNumber, int batchSize) {
        try {
            // load a batch of data rows from file to memory
            List rows = new ArrayList<>(); 
            LineNumberReader rdr = new LineNumberReader(Files.newBufferedReader(path));
            for (String line = null; (line = rdr.readLine()) != null;) {
                int lineNumber = rdr.getLineNumber();
                if (lineNumber >= startLineNumber && lineNumber < startLineNumber + batchSize) {
                    rows.add(line);
                }   
            }   
            
            return rows;
        } catch (IOException e) {
            throw new RuntimeException(e);
        }   
    }   
    
    /**
     * Transform multiple rows of data at a time.
     */
    public static List transform(List rows) {
        List newRows = new ArrayList<>();
        for (String row : rows) {
            newRows.add(row.toUpperCase());
        }   
        
        return newRows;
    }   
    
    public static void main(String[] args) throws IOException {
        // parse command line arguments
        String fileName = args[0];
        Integer batchSize = Integer.parseInt(args[1]);
        
        // partition row indices into batches of given size
        Path filePath = Paths.get(fileName);
        List lineNumbers = CsvBatchProcessor.partition(filePath, batchSize);
        
        // extract data
        Stream stream = lineNumbers.stream().parallel();
        Stream> data = stream.map(i -> CsvBatchProcessor.extract(filePath, i, batchSize));
        
        // transform data
        data = data.map(CsvBatchProcessor::transform);
        
        // load data to console
        data.flatMap(x -> x.stream()).forEach(System.out::println);
    }   
 }   

Streaming and Batching in Java

If you work with external API a lot, you probably have encountered API that supports batch operations.  Instead of making a separate API request for a single record, you can operate on multiple records simultaneously with one batch request.  Computationally, batching helps speed up querying, reduces network overhead and places less load on the remote server.

Batching occurs commonly in the real world in streamlining processes or meeting process constraints.  For example, your favorite pizza chains batches multiple orders into a single dispatch to save delivery costs and time.  An amusement park ride processes guests who are waiting in queue in batches with the batch size being the ride capacity.  The list goes on.

Batching is such a pervasive operation that you will likely encounter the need to write code implementing or simulating the concept in your software development career.  This was the case for me in a recent project.  The requirements involve chunking record IDs and submitting each chunk to an external API for fast look up in Java.  The records IDs are generated in real-time one at a time so the total number of records is not known in advance.

Thankfully, with the addition of streams and lambda expressions in Java 8, the task can't be any simpler.  Without further ado, here's the code that does the trick.

import java.util.List;
import java.util.ArrayList;
import java.util.Spliterator;
import java.util.stream.Stream;
import java.util.stream.StreamSupport;
import java.util.function.Consumer;
import java.io.BufferedReader;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;

/**
 * Read an input file containing one ID per line and process a batch of IDs at once.
 */
public class BatchProcessor {
        public static class BatchSpliterator implements Spliterator<List<String>> {
                private final Spliterator<String> source;
                private final int batchSize;
                private List<String> batch;

                BatchSpliterator(Spliterator<String> lineSpliterator, int batchSize) {
                        this.source = lineSpliterator;
                        this.batchSize = batchSize;
                        this.batch = new ArrayList<>();
                }

                public boolean tryAdvance(Consumer<? super List<String>> action) {
                        boolean success = this.source.tryAdvance(this.batch::add);
                        // flush if batch is filled or input data is exhausted
                        if (success && this.batch.size() == this.batchSize || !success && !this.batch.isEmpty()) {
                                action.accept(this.batch);
                                this.batch = new ArrayList<>();
                                return true;
                        }
                        return success;
                }

                public Spliterator<List<String>> trySplit() {
                        return null;
                }

                public long estimateSize() {
                        return Long.MAX_VALUE;
                }

                public int characteristics() {
                        return ORDERED|NONNULL;
                }

                /**
                 * Divide an input stream of IDs into batches.
                 */
                public static Stream<List<String>> batch(Stream<String> lines, int batchSize, boolean parallel) {
                        return StreamSupport.stream(
                                new BatchSpliterator(lines.spliterator(), batchSize),
                                parallel);
                }

                /**
                 * Process a batch of IDs in one call.
                 */
                public static List<String> processBatch(List<String> batch) {
                        List<String> newBatch = new ArrayList<>();
                        for (String str : batch) {
                                newBatch.add(str + "!");
                        }
                        return newBatch;
                }
        }

        public static void main(String[] args) throws IOException {
                // parse command line arguments
                String fileName = args[0];
                Integer batchSize = Integer.parseInt(args[1]);

                // stream input file
                BufferedReader br = Files.newBufferedReader(Paths.get(fileName));
                Stream<String> stream = br.lines();
                BatchSpliterator.batch(stream, batchSize, true).map(BatchSpliterator::processBatch).forEach(System.out::print);
        }
}

The solution implements the java.util.Spliterator interface and a constructor that accepts a batch size and an incoming source stream consisting of strings.  The key interface methods are:




tryAdvance() - Add the next item from source stream into a batch.  If the batch is full, then pass the batch it to the given Consumer action and initialize a new batch.

trySplit() - Return null because the source stream size is not unknown in advance, therefore, it cannot be split.  Unfortunately, this also means batching cannot be done in parallel.




estimateSize() - Return Long.MAX_VALUE because the source stream size is not unknown in advance.

characteristics() - Return the properties of the Spliterator which in this case involve iterating over an ordered sequence of non-null values.

The other two static methods in the class are our own helper methods.

batch() - Divide an input stream into batches of given size.




processBatch() - Process a batch of strings by appending an exclamation mark to each string.


In the main method, an input file specified in the command line arguments is used as our source stream.  The long chained line of code chunks the input stream of strings, calls processBatch() on each batch, and then dumps each batch out to screen.

BatchSpliterator.batch(stream, batchSize, true).map(BatchSpliterator::processBatch).forEach(System.out::print);


As an example, given an input file with one number per line starting from number 1, the output of the program is:

[1!, 2!, 3!, 4!, 5!, 6!, 7!, 8!, 9!, 10!][11!, 12!, 13!, 14!, 15!, 16!, 17!, 18!, 19!, 20!][21!, 22!, 23!, 24!, 25!, 26!, 27!, 28!, 29!, 30!][31!, 32!, 33!, 34!, 35!, 36!, 37!, 38!, 39!, 40!][41!, 42!, 43!, 44!, 45!, 46!, 47!, 48!, 49!, 50!][51!, 52!, 53!, 54!, 55!, 56!, 57!, 58!, 59!, 60!][16!, 62!, 63!, 64!, 65!, 66!, 67!, 68!, 69!, 70!][71!, 72!, 73!, 74!, 75!, 76!, 77!, 78!, 79!, 80!][81!, 82!, 83!, 84!, 85!, 86!, 87!, 88!, 89!, 90!][91!, 92!, 93!, 94!, 95!, 96!, 97!, 98!, 99!, 100!]
        

From the output, we see that the input numbers are batched in groups of 10 as expected and each number is appended with an exclamation mark as coded in the processBatch() method.  To make use of this code, simply write your own business logic in the method.  Of course, you would also need to change the input stream source according to your use case.