���˿���

I appreciate that acknowledgement. I suggest that £3 million as a percentage of a £100 million budget would warrant further—

Ms Wallace, I was struck by your written submission, in which you said that the processes for implementing the national outcomes are “weak”. Ultimately, are we dancing around the issue? It is good to talk about processes and about agencies, but does it not come down to individuals? Do we need to put people on the spot and make them accountable for delivering things? In our conversations, there is a sense that how individuals and agencies elect to play their part in the national framework is almost voluntary. Do we need cabinet secretaries, ministers, directorates and agencies to report against the framework? Should we make that much more explicit?

Thank you very much. My understanding of the matter is considerably decluttered from when we started.

Is that where an agreement might come into play so that, rather than an outcome or area just being assigned to an individual, there would be agreements that explained the contribution to it?

From looking at the detail that has been presented and from hearing some of the answers today, one of my key concerns is the level of granularity at which the project has been monitored and managed. Mr Carlaw said that it is not possible to specify which bit of spend would have occurred in which year, but I suggest that that is precisely what you should be able to do in a well-run IT project. In a well-run IT project, as I said before, you should understand what levels of effort were required at each phase, and have a project management office that tracks those things, using things such as Gantt charts and projected spend. That seems not to be what we see. I would be grateful if that detail could be provided.

In particular, I am very concerned by the category “non-technical Contractors”, which accounts for almost a third of the total costs. That is a very non-specific category of work. Can you explain why you were using third-party contractors to deliver the project? Managing individual contractors on an individual basis seems to be a very complicated way of managing a project like this, but it seems to be what happened.

Dr French, could you pick up on that and also say whether there is a sense in which the metrics need to be split apart from the capturing of the outcomes? I accept what Dr Elliott is saying, but I think that, if we just had qualitative outcomes with no measurement, we would have a problem. At the focus group that I attended in Glasgow—there were parallel focus groups in Dundee and Glasgow—there was a view that we are not using data properly and that we have narrow metrics, which is a problem because, in the 21st-century world, people use big data sets and do much richer data analysis.

Do we need to split apart the capturing of the outcomes from the measurement, and do we need to overhaul how we conceive of what the measurement looks like so that we can capture that 21st-century big-data approach?

Okay. I could ask more questions, but I do not want to use up more time.

I would be grateful if somebody could write to the committee to clarify a couple of things. A number of boards and management groups have been mentioned; it would be good to get some documentation on how they interrelate and how the governance works.

Secondly, if the agile methodology was used, I would be grateful for an explanation as to why, because it strikes me that the project had a clear functional footprint, so I wonder whether the agile methodology was at all appropriate for delivering a website project such as this one.

Indeed. However, in terms of the overall budget, it is a significant project. In order to understand whether it was getting out of control, we need that level of detail.

However, the problem with transparency goes a little further. On the detail that has been provided by the SPCB about how the costs are accounted for, all that we have been provided with is a schedule of resource costs set out according to whether they were for technical and non-technical contractors, but there is no specificity about what work they were doing.

I would expect, in any IT project, to see phases split up, so that we could understand where efforts are being applied—whether to initial analysis or to the design, build, testing or user acceptance phases, for example. Those are very basic things, but we do not have that level of detail. Why has it not been provided?

I do not understand why, in table 2 of your written submission to the committee, you have described categories as being “areas of spend”, but that is not what they are. The table is a time profile that is broken down by resource type. In order to get a good handle on any IT project, you need to understand that effort by phase, do you not?

I like that phrase “radical decluttering”. It sounds like what I am constantly being told to do at home.

I am very interested in the conversation so far. I will pick up on a couple of points.

Structure seems to be the thread that runs through the conversation at a number of different levels. Dr Ian Elliott talked about the national performance framework originally being a decision-making tool. The outcomes are relatively straightforward to understand. However, when you come to the indicators, you are suddenly landed with a sea of bullet points and it is difficult to see intuitively what they are trying to tell you or even whether it is one thing or a number of different things.

I looked at the indicator on children’s happiness. It turns out that that is just one survey that manages four quite narrow metrics, which are valuable but do not necessarily entirely encompass what we would all understand to be children’s happiness.

Is one of the problems the structure of the NPF, in that we have good high-level outcomes with an asymmetrical set of indicators that sit below them and it is not intuitively easy to understand what any of them tells us? In other words, is the NPF as it is currently structured too difficult to use?