When Intentions Go Public – Does Social Reality Widen the Intention-Behavior Gap? by Peter M. Gollwitzer et al.

On the management of software projects:

Programming management will continue to deserve its current poor reputation for cost and schedule effectiveness until such time as a more complete understanding of the program design process is achieved.

We build systems like the Wright brothers built airplanes — build the whole thing, push it off the cliff, let it crash, and start over again.

Production of large software has become a scare item for management. By reputation it is often an unprofitable morass, costly and unending. This reputation is perhaps deserved.

On the distinction between design and production:

Software production takes us from the result of the design to the program to be executed in the computer. The distinction between design and production is essentially a practical one, imposed by the need for a division of the labor. In fact, there is no essential difference between design and production [...]

The most deadly thing in software is the concept, which almost universally seems to be followed, that you are going to specify what you are going to do, and then do it. And that is where most of our troubles come from.

On dealing with user requirements:

Users are interested in systems requirements and buy systems in that way. But that implies that they are able to say what they want. Most of the users aren’t able to.

We should have feedback from users early in the design process.

On programmer performance (compare with the essay Great Hackers by Paul Graham):

I have a question on the huge range of variability of programmer performance. Are similar ranges found in other engineering areas? [..] The variation range in programming is in fact greater than in other fields.

On the production of software:

We are starting gradually, and building up. My motto is ‘do something small, useful, now.

Large systems must evolve, and cannot be produced all at one time. You must have an initial small core system that works really well.

On testing software:

System testing should be automated as well. A collection of executable programs should be produced and maintained to exercise all parts of the system. [...] As an output of a test validation run, each test should list the modules it has exercised, and as well, should list the interfaces and tables it has tested. It is important to document success, as well as failure.

A lot of people are thinking about how we can reduce household energy use these days. Changing household energy use can be a challenge because to most people there is no working feedback loop available. Energy use is invisible. My own feedback loop consists of getting a monthly invoice displaying the cost I had for the previous month (e.g. “278 SEK”). That is a lousy feedback loop if you want to change behavior. I can not even see how much energy I used. Not that it matters because that information is useless without knowing what would be a reasonable level to compare with. And getting the information once a month does not provide a feedback loop for me to see the effect of my immediate behavior.

A basic model of a feedback loop for a system looks like this:

Based on a desired state (a reference value) a controller operates on a system and adjust control behavior based on the output from a sensor to make the system align the output to the desired state. For this to work there are a couple of things required:

1. We must know what the desired state is. Without it we don’t know if the output from the system is good or bad.
2. We must know the system output. Without the output we can’t calculate the measured error between output and the desired state.
3. The controller must be able to provide system input to control the system and its parts in a predictable way.
4. The delay between the actions of the controller and the sensor feedback needs to be short so that the result of different actions are visible.

Applying this basic model to the typical feedback loop for household energy use we get:

The “system” in this model would be your household. It consists of several things that use energy (refrigerator, TV, other appliances…). You control it by operating various appliances, setting the refrigerator temperature and making purchasing decisions for new hardware that affects energy use. You get the energy bill once a month with the amount of energy used.

As you can see there are several problems. I don’t know the desired energy level. Is my household below or above the average for similar households? If I switch off the TV instead of keeping it on standby, would it matter? Will setting a higher temperature in the refrigerator impact my energy use? Since I can’t see the immediate effects of my behavior there are no drivers to change it.

Real-time feedback and knowledge about the desired state

A first step to  influence behavior would be to provide real-time feedback and information about a desired state. That would make it possible for you to try various control actions and see what implications they have on the system. There are a lot of third party solutions to get real-time feedback e.g. Wattson, TED-5000 and so on, but from what I have seen there are very few options to get information about a desired state. Getting more data about desired state can in part be accomplished by sharing information about your household and its energy use with others.

But, to be able to really affect behavior we need to increase the resolution of the parts of the system (at least sections or types of energy use in the household) by using multiple sensors and comparing the output with multiple desired states. This may quickly become complicated for an individual and a reasonable next step is to automate the controller function (you) by allowing automated control actions on parts of the system (e.g. adjusting your refrigerator temperature based on data about its contents, model, and similar household data).

I guess we have some work to do before we can get a better feedback loop in place. What’s in it for the energy companies? Will we have to install our own real-time meters?

(Thank you to Daniel Brolund who provided interesting insights into control theory last week).

1. The benefits appear when a vocabulary is used.
2. An important factor is how a vocabulary and its parts can be published, discovered and referenced.
3. There needs to be one and only one vocabulary artifact.
4. The vocabulary artifact must be machine processable to allow aggregation and automatic generation of other artifacts.

Background

The need for interoperability arise in scenarios with many contributing actors. In the old days this was solved by hiring a (large) consulting organization that (hopefully) had experience from glueing together many products from their vast product portfolio. Everyone involved was happy if it worked. Being able to swap out a part of this system or connect it to a third party that had a different technical platform was considered a risk.

Today more work is being done but in my experience there are still issues in keeping it all together. Domain experts create terminology in Word documents and architects draw UML class diagrams. They all talk about the domain but each type of artifact does not describe the domain sufficiently. In a different room developers merge domain concepts with coding practices to form software that may express fragments of a domain.

Little work is done to define vocabularies in a way that worked both for software developers and for domain experts. Typically you would find a Word document with a flat terminology that was ignored by the software developers, or, you could find class diagrams in UML or software documentation that was impossible for domain experts to understand. And when you look at the code or interfaces of the system there are no references to the correct item in a vocabulary.

How can we make them work together?

Requirements and challenges

I like to think of semantic interoperability as a state where the same cognitive processes kicks in for two persons studying a specific artefact from the domain they are working in.

In order to work with this in a structured way we need to simplify it. One way to look at interoperability can be found in the draft of the European Interoperability Framework version 2. It introduces four interoperability levels. If we remove the European cross border services context they can be described in more general terms:

1. Technical interoperability: issues involved in linking computer systems and services together (e.g. transport and serialization of data).
2. Semantic interoperability: the meaning of information specified in a way understood by all parties (e.g. definitions, relations and structure of terms used to describe data).
3. Organizational interoperability: coordination of processes in the context where data is used/transformed. (e.g. shared definitions of the roles, responsibilities and interactions of/between participants).
4. Legal interoperability: shared interpretation and understanding of laws regulating information exchange and cooperation (e.g. can we transfer information about people? What about privacy?).

There are other models that could be used to discuss levels of interoperability (e.g. LCIM – Levels of Conceptual Interoperability Model) but let’s stick with the EIF 2 model above.

If you have a scenario with only two parties everything is simple. You could just do the minimum amount of work to make sure systems talk to each other and that people share a common image of the various levels of interoperability. Enabling more parties to participate requires more work though.

If you have a scenario with many parties (where some may not yet exist) you have to create a framework describing how interoperability should work by harmonizing a set of solutions at each level of interoperability. In a dictatorship this is easy. Just evaluate a few solutions, choose one and make everyone use it. In other contexts it is more complicated.

If we consider the EIF 2 model above as a set of requirements we should also make sure that the solutions we propose:

1. scale down: If the parties (organizations, departments etc) vary greatly in size (as they will if the number is sufficiently big) some may consider that the solution you propose in a framework is unnecessary complex. They are right. In order to maximize the number of parties and processes that can participate in an interoperability scenario you will have to make sure the proposed solutions scale down.
2. are technology neutral: Given a large number of organizations you will find almost every technology platform you can imagine. There will also be a number of ways to send and receive information. Some like to force a specific set of transport mechanisms as part of a interoperability framework but this will quickly put you on a path where you can’t make use of new standards and technologies. Forcing participating organizations to use a specific tool or platform will fail.
3. give a reasonable amount of flexibility: This may require the framework to contain partly overlapping standards.
4. facilitate an emerging semantic interoperability: The idea of having a central organization defining everything only works in a dictatorship. We should enable the decentralisation of participating in vocabulary work.
5. are discoverable and referencable: If the artefacts that are created are difficult to find for a newcomer you can give up now. By lowering the threshold to participate more parties can be involved and more information can be reused.

This proposal is aimed at semantic interoperability only.

A lightweight semantic interoperability framework proposal

1. The Web Ontology Language (OWL) is used to express vocabularies

The primary reason for selecting OWL is not the features it has to express domain concepts but rather the possibilities of publishing it and referencing it on the web that come bundled with it. Using the web as a publishing platform enables us to make use of already existing standards for referencing a vocabulary and its parts.

Although plagued by research projects of poor usability and lack of longevity there are several tools to create vocabularies in OWL. The framework does not specify the process for creating a vocabulary as that may hinder small organizations form participating.

2. Vocabularies are published properly on the web

Putting an OWL vocabulary on the web means that it can easily be discovered, referenced and transformed. Not only will it help your own organization but it will lower the threshold for others who want to use parts of it in their own work. As soon as that happens you have the first step towards semantic interoperability.

By “published properly” I mean that you should choose your URI for the vocabulary wisely as it will be the reference string used in software interfaces and other artifacts down the road. Many get web publishing wrong by scrapping one of the best features: long term referenceability, as they dump information tightly coupled to the publishing tool they are using. When you change the tool (e.g. a content management system) you may break references for everyone else. Don’t do that.

To make the vocabulary human readable (nobody wants to read OWL/RDF-schema) generate at least an HTML-version of the vocabulary for users pointing their browser to the URL of the vocabulary and a class hierarchy overview to help users understand relations between things.

3. Use a standardized way of referencing vocabularies in other artifacts

Although OWL may seem to be only for semantic web scenarios it can be used in other areas. By standardizing how references from WSDL and XML schema are made it is possible to keep an unbroken chain of references. SAWSDL provides a mechanism to reference e.g. a class in the vocabulary. This makes it possible for developers to refer to the vocabulary in a consistent way from XML schemas and WSDL files and provides a way to keep the vocabulary closely connected to API implementations.

One important area missing is how to point to vocabularies from code. Code is a design artifact for a system and as such it is important that it is possible to keep an unbroken reference chain to it too. But we’ll leave that open for now.

4. Aggregate all vocabularies on a central website

Although vocabularies can and should be published under a URI in control by the authoring organization they should be aggregated on a central searchable website. This is easy as they are machine readable and will serve two purposes:

1. It helps people discover other vocabularies before creating their own. This is an important step in creating an environment for an emerging semantic interoperability.
2. The aggregated material will become a tool to find areas where harmonization efforts should be focused. By querying the aggregated information it is possible to find vocabularies that define similar things. Multiple classes called “Person”? Get going.

5. Express provenance in the vocabulary

Provenance provides a foundation for assessing authenticity and enabling trust. By standardizing information about provenance you also facilitate interaction between parties involved in the domain.

What do you think?

I believe this could be a lightweight approach to work with semantic interoperability in a way that scales (both up and down) and that provides for an emerging semantic interoperability over time. I guess it could work for government scenarios on country scale as well as for a small organization.

The only thing I miss is a standardized way of referencing a vocabulary from code.

Please leave your feedback.

If you try “:let s = 0.5″ in MacVim and get an error you are affected. This happens when you have launched MacVim from Finder. In this case MacVim does not pick up all the locale settings properly for some reason. This includes the settings for the separator character in floating point numbers which may make all vim scripts with a dot separator (“0.05″) fail.

1. Install the FuzzyFinder script (including the l9 library).

2. In your .gvimrc add the following lines. This will deconnect the Apple-T from File->New tab and instead invoke FuzzyFinder with a wildcard for subdir search:

if has("gui_mac") || has("gui_macvim")
  macmenu &File.New\ Tab     key=<Nop>
  map <silent><D-t> :FufFile **/<CR>
endif

Now, start MacVim from terminal with “mvim” instead of double-clicking the icon in Finder and try Apple-T. FuzzyFinder should now work correctly. Here are some of the discussions for reference:

• Error in vim-l9
• FuzzyFinder and error E806: using Float as a String

On Sunday there were 12 teams presenting their apps. Some of the teams had used public transport data made available by the Viktoria Institute but many were scraping government websites for lack of proper API:s. In the 24 hours set aside for the competition the following apps were created:

• Slagkryssaren: Valkoll – an Android app to see election results where you are right now
• Slumptal: Lokalpolitik – geocoding city council docs to be able to present relevant decisions in your neighborhood.
• New and shiny things: @Myndighetskoll – a twitter bot that ansers questions about government agencies
• Popdevelop: Kollektivt.se – an API service and realtime display of buses in the city of Malmö
• Facebookpolitik: Svenska Riksdagen på Facebook – pushing government decisions and initiatives to Facebook
• First Blood: Engagemangsindex – visualization of government issues
• Boreda: Boreda @ Codemocracy – using public transport data as a selection criteria in finding properties for sale.
• Exjobbarna: CrimeTracker – a Windows mobile app to see crime stats
• Anyro: Tågtider till Android – an adroid app for train data
• Existic: Riksdagsdata i Ply – an extension to PLy to do drill-down in government data.
• Akvo Phone: Georapport – an Android app that geo-encodes and signs a picture that could be used in development aid projects or fix my street-scenarios.
• Kreativ Stuga: Edemokrat.se – An app that connects government documents in a way that makes it easy to navigate them.

More info at the official Codemocracy website. Many CC pictures are in the Codemocracy Flickr group.

I am already looking forward to the next event!

A third recommendation

An important value of public sector information is the increased transparency in government decision making. A European Union directive only affects member states and not EU institutions. We are therefore at risk of missing out on increased transparency in EU institutions that the PSI directive may bring to governments in the member states. Also, government transparency is often considered to be a contributing factor in reducing corruption.

Therefore, it would be great if there was a plan, similar to the PSI directive, that made sure that EU institutions also started providing machine readable data in order to increase transparency and reduce the risk of corruption.

Currently, it seems to be up to each institution to do what they want. Maybe that is why the Publications Office of the European Union decided to charge  for access to electronic versions of the EUR-Lex database (that contains the directives and other legal acts). This move makes it difficult for new actors in the legal information industry to include EU data in their products which leads to reduced competition.

Use of social media tools in the public sector is in its infancy in Sweden. In a study conducted by the Stockholm Chamber of Commerce in 2009 (PDF) , around 12% of agencies reported that they were using social media channels in their communication. In another study by the Swedish Public Relations Association 75% of PR managers were evaluating the use of social media but only 5% had involved social media in the communication strategy. Many agencies are trying out new tools to see how they fit in their communication strategy. Most are using them as a news feed (one-way communication) and do not reply to questions. The number of followers varies greatly. Some examples:

Some agencies have also started using Facebook. The National Heritage Board (Riksantikvarieämbetet) on Facebook has close to 1200 fans. They have also used Flickr to increase access to parts of their archives. Currently the images are low-res only but that may change in the future.

Municipalities on Twitter

Many municipalities have started using Twitter to push information to citizens. Most are using it as a one-way communication channel, but some are engaging in dialogue. The number of municipalities on twitter is increasing almost on a daily basis. In my list of swedish municipalities on twitter there are currently 71 (out of 290). Many of them are not publishing any tweets and may only have registered the account to get a proper twitter handle.

In he following visualization you get an overview of how the municipalities compare with regards to number of tweets, followers and “early adopter”-status (“Age” as in number of days on twitter). Uppsala (north of Stockholm) has over 500 followers and published 941 tweets in their 367 days on Twitter. See here for a larger chart.

Observations

Some organizations are following citizens on Twitter and may not have realized that people may feel uncomfortable having an agency following them.

There are also some fake agencies on twitter. Swedish national authority for signals intelligence, FRA, has a twitter feed (with the correct logo), but looking at the tweets it is likely a fake. How should agencies relate to this? The police has several twitter feeds it seems. But they are not managed by the agency itself. How can citizens trust the information in these channels when it is difficult to see if they are managed by the agency or some personal initiative?

Recommendations from the government

On March 23 the government published a proposal saying (my translation):

Social media tools create new possibilites for communication with citizens and businesses. The E-government delegation will create guidelines for the public sector use of social media tools.

This means that we will likely see more agencies trying out social media channels in the future.

Do you have more information? Please comment!

I will be working with IT standardisation issues and the next revision of guidelines for public websites.

Like most people living in Stockholm (and other cities I guess) I live in an apartment building to which you have no access. Also, like most people, I work during the day. Nothing special, just normal office hours, about the same hours when your drivers drive around the city trying to deliver stuff to the empty homes of people who are at work.

Knowing that you will fail to enter the building where I live (it doesn’t really matter since I’m not at home anyway), I have tried to contact you to see if I can do a part of your work by going to your terminal myself to pick up my package.

For some unknown reason you can’t refrain from trying to deliver it my empty home. Maybe it is in he nature of logistics people inherited in genes from loyal post office workers of the good old days?

Anyway, this means I have to wait for you to fail delivery and take the package back to your terminal and then send me a message (through regular mail) that you failed. A process that takes several days.

When I walk home from work I pass a 7-eleven with a sign that says “DHL Servicepoint”. It is bright and yellow and only about 150 yards from my home (I can almost see it if I lean out through my living room window). Intrigued by the words “service” and “point” I inquired, though your eminent call center, if it was possible to have the package delivered there instead? I was hoping I could save both you and me some work (and time). For unknown reasons you are not allowed to deliver my package to your own servicepoint. Words fail me.

My second suggestion was to change the address so that you may deliver it to my workplace instead. It is a bit of a gamble since I am in meetings throughout the day and may not be around at the exact random time your delivery guy tends to show up. This was possible, your friendly call center employee told me, but we have to try to deliver the package to your empty home first.

How hard can it be? I had a crazy idea. What if logistics was all about making use of information to optimize flows of goods?
Visa DHL delivery på en större karta

The myth of multitasking:

… [a] research study, funded by Hewlett-Packard and conducted by the Institute of Psychiatry at the University of London, that found, “Workers distracted by e-mail and phone calls suffer a fall in IQ more than twice that found in marijuana smokers.”

Stanford study: Cognitive control in media multitaskers:

Results showed that heavy media multitaskers are more susceptible to interference from irrelevant environmental stimuli and from irrelevant representations in memory. This led to the surprising result that heavy media multitaskers performed worse on a test of task-switching ability, likely due to reduced ability to filter out interference from the irrelevant task set.

Meet the Life Hackers – New York Times Magazine (10/16/05) (based on research by Gloria Mark, University of California)

Research shows that 40 percent of the time, workers ramble along a different tangent when an interruption ends because their short-term memory has been disrupted.

The Internet is the public space of the modern world, and through it governments now have the opportunity to better understand the needs of their citizens and citizens may participate more fully in their government. Information becomes more valuable as it is shared, less valuable as it is hoarded. Open data promotes increased civil discourse, improved public welfare, and a more efficient use of public resources.

…and this is why I have launched opengov.se – an initiative to bring swedish public sector data online in open formats. Initially the website contains a catalog of public data sources and information about licensing and formats used.

You can help out by suggesting datasets form the swedish public sector.

class Organization(models.Model):
    name = models.CharField(max_length=255)
    url = models.URLField(verify_exists=False)
    orgtype = models.ForeignKey(OrgType)

The same model class in Rails typically looks like this:

class Organization < ActiveRecord::Base
    belongs_to :OrgType
end

…and in Grails it is more specific like Django:

class Organization {
  String name
  String url
  static belongsTo = OrgType
  OrgType orgtype
}

It took me a while to remember that in Rails, parts of the model design is actually stored in the database schema instead of in the Ruby code. Peculiar don’t you think, given that everything else in a Rails app is nicely declared in Ruby code? There are of course benefits to both approaches, but I have started adding comments in the Rails model classes to be able to remember what properties they have without peeking in the Db. Typically I have a number of half-baked projects on my laptop and from time to time I forget what they do and these comments help me remember.

Check out more examples here:

• Django models
• Rails models
• Grails models

Problem 1

If we list all the natural numbers below 10 that are multiples of 3 or 5, we get 3, 5, 6 and 9. The sum of these multiples is 23. Find the sum of all the multiples of 3 or 5 below 1000.

Brute force version below.

n = 1
sum = 0
while(n < 1000,
      if(n % 3 == 0 or n % 5 == 0, sum += n)
      n++
)
sum println

Or, as a one-liner after a suggestion from Ola:

(1..999) select(n, n % 3 == 0 or n % 5 == 0) fold(+) println

Problem 2

Each new term in the Fibonacci sequence is generated by adding the previous two terms. By starting with 1 and 2, the first 10 terms will be: 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, … Find the sum of all the even-valued terms in the sequence which do not exceed four million.

sum = 0
fibs = [0, 1]
current_fib = 0
while(current_fib < 4000000,
    current_fib = fibs[0] + fibs[1]
    if(current_fib % 2 == 0, sum += current_fib)
    fibs[0] = fibs[1]
    fibs[1] = current_fib
)
sum println

Problem 3

The prime factors of 13195 are 5, 7, 13 and 29. What is the largest prime factor of the number 600851475143?

Using brute force for this one. Faster and more elegant would have been the Rho algorithm.

start = 600851475143
num = start
factor = 2

while(factor * factor <= num,
    if(num % factor == 0,
        factor println
        num = num / factor,
        factor++
    )
)

if(num != 1, num println)

Problem 5

2520 is the smallest number that can be divided by each of the numbers  from 1 to 10 without any remainder. What is the smallest number that is evenly divisible by all of the numbers from 1 to 20?

First, as the numbers 1 to 10 are factors in the numbers 11 to 20 we only need to check divisibility for the the latter. Starting of by setting up methods for greatest common divisor and least common multiple. Brute forcing this by looping over an incremented number will not work in the current version of Ioke (takes a couple of hours). Brute forcing it in Ruby took a couple of seconds.

gcd = method(a, b,
    if(b == 0, a,
        gcd(b, a % b)
    )
)

lcm = method(a, b,
    (a / gcd(a, b)) * b
)

(11..20) inject(number, n, lcm(number, n)) println

Problem 18

Find the maximum total from top to bottom of the triangle below. This solution also works for problem 67 (a much bigger triangle).

By moving from bottom to top, calculating each cell’s maximum sum and replacing the value with it we’ll end up with the total in the first cell in the triangle.

triangle = [
[75],
[95, 64],
[17, 47, 82],
[18, 35, 87, 10],
[20, 04, 82, 47, 65],
[19, 01, 23, 75, 03, 34],
[88, 02, 77, 73, 07, 63, 67],
[99, 65, 04, 28, 06, 16, 70, 92],
[41, 41, 26, 56, 83, 40, 80, 70, 33],
[41, 48, 72, 33, 47, 32, 37, 16, 94, 29],
[53, 71, 44, 65, 25, 43, 91, 52, 97, 51, 14],
[70, 11, 33, 28, 77, 73, 17, 78, 39, 68, 17, 57],
[91, 71, 52, 38, 17, 14, 91, 43, 58, 50, 27, 29, 48],
[63, 66, 04, 68, 89, 53, 67, 30, 73, 16, 69, 87, 40, 31],
[04, 62, 98, 27, 23, 09, 70, 98, 73, 93, 38, 53, 60, 04, 23],
]

;; We replace cells from the bottom up by finding the max sum for each
;; position in the triangle.

current_row = triangle length - 2

while(current_row > -1,
    pos = 0
    while(pos < triangle[current_row] length,
        triangle[current_row][pos] = triangle[current_row][pos] + \
        ([triangle[current_row + 1][pos], triangle[current_row + 1][pos + 1]] sort)[1]
        pos++
    )

    current_row = current_row - 1
)

triangle[0][0] println