On a mission to create a violin for the 21st century, Keith McMillen developed a smart fabric that can provide humanoid robots with tactile awareness that exceeds the capabilities of human beings with respect to spatial resolution and sensitivity.

Sensors are fascinating. They are great magicians turning one thing into another. Estimates say the number of sensors on Earth is approaching 1 trillion devices. Hundreds for each of us!

As a kid growing up in Chicago, my first sensor experiments targeted the home thermostat: a flat snail of metal attached to a glass bottle of mercury. As soon as I could reach it, I was in control of that monster of an oil furnace with its arms reaching up from the basement into our home.

The thermostat is considered by many to be the first sensor. Made in 1883 by a guy who still has his company named after him: Johnson Controls.  Take that Tom Edison.

When I was nine, my big brother gave me his old Sears electric guitar. Unwilling to leave anything alone, my younger brother’s record player got turned into a guitar amp. I wanted to start a band, but I could tell by looking at TV you need a singer and a microphone. Guess what? A speaker is a decent microphone. 

Reading “Here, There and Everywhere” last year, I learned that the great Geoff Emerick was using a speaker to capture Paul McCartney’s bass at the exact same time. Used on “Rain” and “Paperback Writer” in 1966, the recording genius beefed up Paul’s bass sound to great effect by using a speaker plugged into the board at EMI. Outclassed, I’m glad we disbanded.

Repairing musical instruments paid for my degree in acoustics from U of I in Urbana. Every musician since Beethoven wants to be louder. Actually, the Greeks started the volume wars in 300BC with really loud keyboard organs invented by Ctesibius of Alexandria. Used later by the Romans at the Coliseum.

 Every acoustic guitarist, violinist and percussionist who wanted to be in a rock band needed a pickup. These were usually single element piezo electric lumps mounted in bridges and bodies. Sounded horrible.

I started making my own violins through my company, Zeta Violins, with a dual element pickup – per each string. A violin string vibrates in the direction of bowing but makes a scraping sound as it returns across the horsehair ready to be pulled again. Stick and slip. The two piezo elements were in a “V” configuration which amplified sound in the horizontal plane but rejected the vertical bow noise.

Zeta Violins grew in popularity, I sold the company to Gibson, and they are still in production today. But I was not done with the bowed strings.

With the growth of computers on stage capable of many times the processing of all of EMI studios, string players wanted more access and expressive possibilities. So, I created the K-Bow, a Bluetooth sensor bow. Tracking X-Y-Z motion with an accelerometer, bow placement with optical and RF sensors it called out for a grip sensor. The violin bow is held by the “stick” in a leather covered area in front of the “frog” called the “grip”. 

A very intimate cylinder of interface between the player and the played. Now how was I to make an extremely lightweight reliable cylindrical sensor that could be mounted to permanently and be expected to work for decades?

This began my research into fabric sensing. A small piece of non-woven material with conductive particles bonded to the surface of each fiber gave promising results. The outer surface of the fibers of this wildly entangled felt fabric would change resistance as they were compressed. The basic principle was sound. Proof of concept.

It took 3 years of additional work to make this concept into a real sensor through KMI.  The conductive particles would not stay in place, coming off on your hand during assembly. Solved.

Changes with time and use? Now stable over millions of cycles of 75-pound hits. (I have sensors out there for over a decade still meeting spec).

Dynamic range? Greater than 80 dB (10,000 to 1). Depending on deployment, measurements from 5 grams to 150 Kg all from the same fabric. Frequency response is flat to 1000 Hz and then rolls off at 3dB per octave.

An easily corrected smooth temperature co-efficient – always a good thing. Spatial resolution – from 1 mm to 10 cm. Tight variable geometry curvature with 1.5mm radii. 

You need to source a current into a point on the fabric and measure a voltage nearby to see the piezo resistive behavior. Many methods evolved. 

My favorite? Screening conductive inks on both sides of a 100-micron thick TPU (Thermoplastic Poly Urethane) membrane. Add laser punched holes providing vias we now have a double-sided circuit that is waterproof, extremely flexible, and vanishingly thin. Just melt it onto the fabric.

Fabric is a versatile material. Depending on how you mount it you can perceive pressure and location in 3D, bend accurate to minutes, and shear forces. It is possible to do all three with the same piece of fabric. 

After the K-Bow, KMI Music proceeded to make a dozen instruments all using smart fabric sensors. Over the last decade, instruments with over 4M sensors are out there in regular use.

We say there are 5 levels of electronic reliability: consumer, commercial, military, NASA and Rock and Roll. 

And musicians are hard on their gear. I am blessed with an incredible team of engineers who are artists and musicians. Most have been with me for 8 – 10 years and they carry a sense of pride, knowledge and demands for low latency and repeatability. This is required for a musical instrument that can be loved and depended on.

Due to demand for non-musical devices, we spun out BeBop Sensors, Inc, the world leader in smart fabric sensor technologies with millions of sensors in daily use and 30 patents.

Initial designs included soles for exoskeletons and for human movement and activity. Fabric is comfortable, non-threatening (as are cameras in certain applications) and can sense detail in places where no other sensor could survive.

Such as inside a person’s shoe. Now is a good time to talk about the ubiquitous FSR – Force Sensing Resistor. These have been around in one form or another for about 35 years. An FSR consists of two sheets of film, usually PET, with traces on one piece and a conductive material on the other. They face each other and when compressed resistance changes. 

There are some applications where this is a fine sensor. It really prefers to be flat and mechanically well supported. Dynamic range is mid 40 dB and they change over time. The inside of a shoe is not the kind of place for FSRs.

Many of our customers have tried to make FSRs behave but all have failed in one requirement or another. There is a lot of fabric in most shoes so why not a little more. 

A wonderful challenge for BeBop was making sensors for robots – RoboSkin. On a human like finger, we can sense 80 locations with as little as 2 mm pitch. This is better than human spatial resolution. Forces from 5 grams to 50Kg are reported by each of the 80 sensors. 

We welcome our robotic helpers. When Ricky Gervais was asked if he feared being ruled by artificial intelligence, his reply: “I would love to be ruled by any intelligence.”