Two minor things to note:

• They quoted me saying “desegregation“, when I actually said “disaggregation“. After looking up the definition of the former, it’s not awfully far.
• The article, in the third paragraph, somewhat overestimates the time we’ve spent working on this (e.g., I haven’t been here 4 years yet).

The goal of this project is to develop simple and inexpensive solutions for obtaining appliance-specific electricity consumption information from easy-to-obtain data sources like the overall power of the building. It’s the idea for which we recently received $1.5M from the National Science Foundation.

We attempt to detect and correctly identify state-transitions for individual appliances in a building (e.g., a light-bulb going from state “off” to state “on”), by carefully analyzing the overall voltage and current signals for the building’s electrical distribution system. We also explore how different statistical and machine learning tools can be applied to solve the problem.

NILM has been around for more than two decades, but mainly as a research topic. What we really want to do is to take it a step further and find out what are the obstacles that we need to overcome to bring it to homes.

I will be posting more about it now that I finally found a quick way to introduce it to all of you (i.e., the video). There are also some interesting links to the other project I am working on: Sensor Andrew.

After doing a little research and tweaking, I solved it by compiling it from source, and following the steps below:

• 1) Modify your configure.in file, by adding:
  KERNEL_HEADERS=/usr/src/kernels/your_current_kernel
  And substituting “your_current_kernel” with yours (e.g. 2.6.18-164.11.1.el5xen-x86_64).
• 2) Place that line of code, at around line 405, right before:
  if test -z "$KERNEL_HEADERS";
• 3) From the shell, run the following to complete the installation:
  autoconf
./configure
make
sudo make install

Algunas notas:
1) ¡Qué largo! Ya entiendo por qué estaba tan sediento al terminar la charla.
2) Me impresiona lo mucho que siento que he mejorado el inglés después de tres años aquí.
3) Hay muchas cosas que quiero mejorar la próxima vez que tenga que hacer esta presentación. Por ejemplo, muchos de los “slides” van en contra de mi política actual que limita el número de palabras en cada uno a aproximadamente 15 ó 20.

• dataTimeStamp, as a DATETIME type.
• fractionalTimeStamp, as a DECIMAL(6,6) type.

The first attribute would store the timeStamp with a precision of up to a second, and then the other attribute would store the fractional values. This clearly works, but is far from being optimal. Specially when you have a huge dataset (days or weeks of continuous data) and want to extract a specific region in time.

As a quick solution, I decided to consolidate those two fields into a single BIGINT type column (I’m using field, column and attribute interchangeably here). The new column would contain a 64-bit integer with the first 32-bits used for a traditional UNIX timestamp (number of seconds since 00:00:00 1/1/1970, also known as the epoch), and the other 32-bits would be used for the fractional part of the timestamp, only this time in nano-seconds (10^-9).

To do this conversion, I’m thinking about using something like:

UPDATE power_data
SET timeStamp =
(CAST(UNIX_TIMESTAMP(dataTimeStamp) as UNSIGNED) << 32 | CAST(fractionalTimeStamp * pow(10,9) AS UNSIGNED));

I'm still not sure if this is such a great idea. On the positive side, this new column should sort the table well (instead of having to sort by the two previous columns). On the negative side, I would still have to convert those numbers back into meaningful dates whenever I want to use them.

Another option would be to use the full 64 bits to encode the number of nano-seconds since the epoch. This would be OK for the next 500+ years, since 2^64 is approximately 1.85 x 10^19, and the number of nano-seconds since the epoch on the year 2510 is approximately 1.57 x 10^19. This approach would also sort well, and would need a less complicated parsing process in the end.

What do you think? I think I'm going to go with the second option.

Sensor Andrew, one of the projects I’ve been working on for the past two years, is an experiment on the interoperability and semantic capabilities of cyber-physical systems. In essence, it’s a collection of hardware and software elements that together form a virtual instrument for large-scale sensing and actuation. But I don’t want to spend time explaining it. The interested reader can refer to our webpage or the techincal report.

What I do want to talk about is the relative ease with which new applications, that make use of the available sensor data, can be created.

Today, for example, I decided to write a simple program that would notify me via e-mail if a sensor has stopped reporting for longer than it should. This is a farily common problem as you may expect. There were two ways I could think of easily implementing this. But before I describe them, let me first provide you with a little bit of context.

Sensor Andrew allows any sensor or actuator that can be connected to an Internet-enabled computing device, to be discovered, shared and re-used. That is the basic principle. The best way to explain it, however, is by using an example. Let’s say you just purchased a new home energy meter like “The Energy Detective” (TED). You notice that the display has a USB port on the side, and you wonder if you can use it to access the data and play with it. You go to the Sensor Andrew website, look for the available software adapters (code that is created for this purpose, using our libraries) trying to find if someone has created one for the TED. You find an adapter that fits your operating system and hardware, and proceed to set it up. After registering your sensor with Sensor Andrew (which involves creating an account, describing the sensor’s make and model, the update rate, its location, what it is measuring, etc.) you are given a unique identifier which you will use when publishing data, and an “Event-Node” to which the sensor will be publishing. The event node will be something like “myTED@sensor.andrew.cmu.edu”.

To make the long story short, you can now configure the adapter you downloaded with the information you were given (account, Event-Node, etc.), connect the TED to your computer using a USB cable, and run it. The data from your TED will be periodically published, using a standard XML format, to an internet-addressable resource (the event-node myTED@sensor.andrew.cmu.edu). Behind the scenes, Sensor Andrew is leveraging the eXtensible Messaging and Presence Protocol (XMPP), which you have most probably used even without realizing it. XMPP is the backbone of Google Talk, for example. More especifically, we created a Sensor Over XMPP (SOX) library, that uses the XMPP PubSub (publish and subscribe) protocol extension for this purpose.

I think that this may be enough context. Let me return to the main point of this post. So, now imagine that for some reason your sensor data stops being published. What do you do? How can you tell? Given that any user with appropriate permissions can subscribe to myTED@sensor.andrew.cmu.edu, we can create a simple software agent that subscribes to this event-node, and notifies the user if there has been no message published in the last x minutes. The SOX library, in this case the C library, will help us do this by using only a few functions, namely:

1) Creating the connection:

xmpp_connection = start_xmpp_client (username,
                  password,
                  xmpp_server,
                  xmpp_server_port,
                  xmpp_ssl_fingerprint, pubsub_server, NULL);

2) Subscribing to the event-node:

subscribe_to_node(xmpp_connection, event_node);

3) Then parsing the incoming XML in some way (using expat here), to catch a significant change in timestamps that may indicate a fault, and then trigger an alarm:

static void XMLCALL startElement(void *data,
                  const char *element_name,
                  const char **attr) {

...

   if(strcmp(element_name,"TransducerValue")==0) {
    for(i=0;attr[i];i+=2) {
      const char* attr_name = attr[i];
      if(strcmp(attr_name,"timestamp")==0)
        sensor_timestamp = attr[i+1];
    }

    timestamp_sec_new = convert_to_seconds(sensor_timestamp); 

    if(timestamp_sec_old) {
      if(timestamp_sec_new - timestamp_sec_old > typical_interval)
         alarm(event_node, email);
    }

    timestamp_sec_old = timestamp_sec_new;

    ...

}

Obviously I am ommitting some of the details, and the code is not at all commented, but I guess you can get the point. The alarm function will basically send out an e-mail informing me that my event-node has stopped publishing for longer than the typical_interval.

Tenía US$400 que la universidad me había dado para estos fines, así que decidí gastarlos todos de un solo tiro. Aquí está el carrito de compras:

• Spikes: Exploring the Neural Code (Computational Neuroscience) – Fred Rieke; Hardcover
• Handbook of Statistical Analysis and Data Mining Applications – Robert Nisbet; Hardcover
• The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition (Springer Series in Statistics) – Trevor Hastie; Hardcover
• All of Statistics: A Concise Course in Statistical Inference (Springer Texts in Statistics) – Larry Wasserman; Hardcover
• Introduction to Linear Algebra, Fourth Edition – Gilbert Strang; Hardcover
• All of Nonparametric Statistics (Springer Texts in Statistics) – Larry Wasserman; Hardcover

El mismo Wasserman está dando la clase de “Statistical Machine Learning” aquí.

El “Dr. Mario Bergés”. Bueno…

El caso es que ahora toca pensar en cuál será el próximo paso. De hecho, como los americanos son muy buenos en eso de hacer planes a largo plazo, hace tiempo que debí estar pensando en cuál será el próximo paso. Las solicitudes de empleo para una posición de profesor de universidad se hacen con 8-12 meses de anticipación. O sea, que si quiero solicitar para el semestre de otoño 2010, ya estoy un poco tarde. Pero ese no es el único camino.

En el plano más simple, tengo tres opciones:

• El camino universitario (Academia)
• El camino comercial (Industry)
• El camino independiente (Start-up)

De esas tres actualmente la primera me llama más la atención, pero las otras dos pueden ser igual de buenas. Por otro lado, quisiera que mi próximo paso no me lleve lejos de Pittsburgh ya que a Laura le quedan 2.5 años más de residencia médica. Por suerte hay una cuarta opción que es quedarme trabajando en el mismo proyecto en el que estoy trabajando ahora, [either] como Post-doc, o como estudiante de doctorado si decido no defender la tesis este año. Esa cuarta opción es el comodín, y es la que mantiene mi calma.

Entonces, ¿qué pienso hacer?

Aunque se nos han acercado algunos inversionistas interesados en crear una compañía en torno al tema que estoy desarrollando para mi tesis, los vínculos con Bosch (la compañía que estuvo financiando el proyecto) no han ayudado a que eso se materialice. De modo que pongamos a un lado la opción de un Start-Up.

El camino comercial tendría que ser aquí en Pittsburgh, así que tal vez solicite en Google, la misma Bosch, y algún otro sitio. Pero no es mi opción favorita.

Aún estando tarde para solicitar en las universidades, pienso hacerlo. Como mínimo, voy a enviar mi solicitud a CMU porque tienen una posición abierta en mi departamento. Las solicitudes para esa posición van a empezar a ser revisadas el 1ro de Enero, 2010. Es decir, este próximo viernes.

Entonces, ¿qué estoy haciendo?
Trabajando como un loco para ver si consigo completar los documentos necesarios.
Entre ayer y hoy estuve trabajando en mi Curriculum Vitae. Tengo un primer borrador. Son bienvenidos los comentarios.
Todavía me falta escribir un “Research Statement” (1 página), un “Teaching Statement” (1 página), y un “Cover Letter” (1 página más).

El 2010 se acerca, y yo sigo trabajando.

El complejo está repartido en varias zonas al norte de la isla de Manhattan, en NY. La edificación en la que nos tocó dar la charla y pasarnos el día, queda en Yorktown Heights, y aunque fue construida en los 60s, la arquitectura sigue siendo moderna e imponente:

Los temas de investigación son muy variados: desde cosmología hasta almacenamiento de energía; y aún así el ambiente de trabajo en el edificio deja una extraña sensación de demasiada estabilidad, orden, poco caos. No me imagino un centro así exisitiendo en Rep. Dominicana, por ejemplo.

A la charla asistieron, físicamente, unas quince personas, y un número desconocido adicional por videoconferencia y teléfono. La charla nos quedó bien, pero dada las condiciones de estabilidad y orden que exhume el entorno, no creo que haya podido quedar mal. Todo durante el día transcurrió sin fallas, las cosas se sucedían como si hubiesen sido ensayadas ya. Cada reunión que tuvimos fue seguida por la próxima en la agenda, salvo ligeras modificaciones. Los investigadores son bien remunerados allí. La civilización (la ciudad de Nueva York, por ejemplo) está por lo menos a 20 millas y separándolos está el bosque. La comida está lista, las ventanas limpias, las luces todas encendidas, los baños uno en cada piso, no hay goteras, la gente sonríe…

En fin, me quedó siempre detrás la pregunta: ¿podría yo vivir así, tan lejos del caos?

Idealmente sería una persona con experiencia en programación web y con cierta pasión por temas medio-ambientales y de “sustainability” (no encontŕe traducción al Español). Para la opción de doctorado, todos los costes estarían cubiertos.

El proyecto está relacionado, entre otras, con las siguientes cosas:
http://research.stepgreen.org/
http://sensor.andrew.cmu.edu/about/