Japan’s Tentative Science Budget for 2009
Japan’s new Prime Minister, Taro Aso, participated in his first meeting of the Council for Science and Technology Policy (CSTP). The CSTP is the major decision-making body for science and technology within Japan’s Cabinet Office.
The meeting on 31 Oct. discussed the annual budget for science and technology for 2009 and the important issues which should be prioritized within the budget. With a requested increase of 14% for the overall science and technology budget, greater importance is being placed on areas of research such as innovative technologies, environmental energy technologies, science and technology diplomacy, activation of science and technology in the regions, and projects for the solution of social problems. Overall the budgetary requests for these areas increased by 43% on last year.
The budget requests were received from each Government Ministry in August 2008. The Cabinet will decide on the 2009 budget in December. The Diet, Japan’s Parliament will vote on the budget before 1 April 2009.
Friday, December 19, 2008
Monday, December 15, 2008
So does this mean that people should not talk about their experiences during job searches? Actually the message seems rather more general to me. It is not an example of the risks of blogging a job search so much as the risk of blogging under the cover of anonymity - or rather, the presumption of anonymity. It provides a prime illustration of one of the main reasons why I don't blog anonymously - I would much rather write in the knowledge that anyone might read this, and choose my words appropriately for a public discussion, than sound off under the presumption of anonymity and then suffer the embarrassment of being uncovered. I do accept that some people may have good reasons for blogging anonymously, but am somewhat concerned that it appears to be predominantly female bloggers who feel the need for this cover (perhaps my perception on this is skewed, so if anyone has some hard numbers they would be welcome). But anonymous or not, it is always necessary to consider how people might react when they read what I write, and in fact if I blogged anonymously I would probably have to conceal more (to protect my identity) than I do knowing that I'm out in the open. Even though the typical institute atmosphere is perhaps slightly more "corporate" than a university setting, we are still officially encouraged to present our work to the public and I would not be comfortable working somewhere I was not expected to offer my opinions in public. In principle, many big scientific meetings are open to the public - at least on payment of a fee, and sometimes journalists also attend.
Getting back to the topic of job searches, it seems to me that the main reason for keeping quiet would not be related to offending the potential new employers, but rather so as to not alert the current one, especially if the blogger in a vulnerable position. (Of course if the current position was a term-limited post-doc with no chance of extension that would not apply.) The only time that have I told a boss I was planning on leaving (and why) before actually having a new job lined up, his response was to ask for jules' CV in order to see if he could find a suitable position for her...
Tuesday, December 09, 2008
Based on the talks I attended, the field basically consists of little more than people data-mining for correlations that usually fail to hold when tested on out of sample data, and for which there is no real evidence or even plausible modelling to support the hypothesis that there may be a significant effect. Of course it is obvious enough that gross changes in solar output will have some sort of effect, such as the orbital changes at Milankovitch time scales, and a modest sunspot-related influence. But nothing of much significance in the context of anthropogenic global warming. The main speaker (Usoskin) even admitted as much, despite doing his best to present (in his own words) somewhat of an advocacy of the hypotheses rather than an impartial review. I mostly went along to see if there would be any sceptic-baiting fun to be had (remembering that Akasofu had been a previous invitee here, and having heard some vague rumours of scepticism in the higher echelons of Jamstec itself) but was rather disappointed by how weakly it was all presented. The only really interesting point was that Jamstec has apparently decided in its wisdom to sponsor a new big interdisciplinary project in this area.
To be fair, I don't know the exact scale of this project, although one person presented results from a high-resolution model that used fully half of the Earth Simulator to run, which certainly didn't come cheap. And I do think that it is worth providing a certain amount of support for somewhat speculative and innovative research - 5 years ago people told me that my ideas wouldn't work either. But I'd expect the resources could make a more useful contribution if they were focussed on climate science rather than being siphoned off on this wild goose chase.
Thursday, December 04, 2008
Japan aims to internationalize its science enterprise - Physics Today December 2008
Some choice excerpts:
In a bid to attract both global recognition and foreign scientists, last year Japan launched the World Premier International Research Center Initiative, or WPI. For 10 years, the Ministry of Education, Culture, Sports, Science and Technology (MEXT) will pour ¥ 7 billion (roughly $70 million) annually into five new interdisciplinary institutes in cosmology, materials science, nanoscience, immunology, and the interface of cells and materials.
The WPI institutes are supposed to have about 30% foreigners among their researchers. That, says Okaya, "is bizarre for Japanese. The WPI is revolutionary. It's innovative and very flamboyant."
The other WPI institutes are the Immunology Frontier Research Center at Osaka University, the Advanced Institute for Materials Research at Tohoku University, and the Institute for Integrated Cell–Material Sciences at Kyoto University. All five have different formats, and interactions with their host institutions vary. The common features, which were in part set by MEXT, include using the MEXT funding for salaries and start-up funds, aiming for a total of about 200 people per institute, setting a minimum number of non-Japanese members, and raising additional funds from other sources. Host institutions are expected to provide buildings and other resources. IPMU, for example, is getting a new building and two positions from the University of Tokyo. The MEXT money may be extended to a total of 15 years.
English is the lingua franca at the WPI institutes. And, to attract people to them, the seniority-based pay scale typical in Japanese universities has been turned on its head. For example, says Okaya, the director of IPMU earns more than the president of the University of Tokyo. More broadly, salaries at the institute are higher than professors typically earn in Japan, says Murayama. "We pay better to compensate for people [from Japanese universities] losing their pension plans" and to attract foreign scientists.
"To my pleasant surprise, people in their thirties gave up tenured jobs" to come to IPMU, says Murayama. "Because this place cannot offer tenure, the hardest generation to get is in the forties and fifties. Thirties is easier—they are ambitious, they think this is a place they can concentrate on research for 10 years and then go wherever they want. The forties and fifties think ahead, and might be worried about finding another good job. In the late fifties it's easier, because in 10 years they will retire anyway."
Sounds like an exciting and novel idea? Allow me to introduce the MEXT-funded Frontier Research Centre for Global Change, which was set up more than a decade ago as a highly internationalised and collaborative institute for research into climate change. It had a substantial number of foreign scientists, English was the lingua franca (it was an explicit requirement of employment when I applied here) and they even interviewed some internationally-recognised scientists for senior positions, although they didn't quite get round to employing them (ignoring the case of Suki Manabe who is clearly regarded as Japanese still even though he took US citizenship years ago).
Of course just as is mentioned in this new article, the short-term nature of our employment system means that no decent mid-career Japanese scientists were prepared to come and work here. Instead they act as part-time managers who turn up about once a month but understandably focus most of their attention on their real jobs elsewhere, and this legacy of absentee management remains to this day. This keeps the institute weak which probably suits all the major domestic rivals (or as the Japanese say, "collaborators") as well as our overlord Jamstec which doesn't really understand, or see it as its mission to get involved in, scientific research (and certainly not climate research). The terms and conditions of employment are still good in parts but have been substantially degraded for many, especially the more junior employees. Now Japanese fluency is explicitly required for any senior positions, the institute has been more-or-less fully absorbed as a domestic Jamstec research centre and the "Let's Internationally" theme has been basically replaced with "Ganbare Nippon". Another fortunate consequence of (and indeed perhaps a major motivation for) the post-doc style term-limited contracts is that the foreigners can be easily sent home (or will leave voluntarily) when they have fulfilled their purpose of raising the profile of these new institutes. Of course there are still some foreign hangers-on here who haven't left yet but our proportion is significantly down on the early years.
I confidently predict the evolution of these new institutes will follow in our path - they have even re-used the "Frontier Research Centre" name (which incidentally we are losing next year as part of our ultimate absorption) for one of them!
Back to the closing paragraph of the article:
Mark Vagins is in his early forties, but he jumped at the offer to move to IPMU. He'd been shuttling back and forth between the SuperKamiokande neutrino detector in Japan and a soft-money position at UC Irvine for years. "I have long believed that discoveries tend to get made where fields collide. It's very unusual to have pure math people interact with people who build experiments," says Vagins, who hopes to increase SuperKamiokande's sensitivity by adding gadolinium salt to the water to make neutrons visible in a project called GADZOOKS! (gadolinium antineutrino detector zealously outperforming old kamiokande, super!). "My guess," he adds, "is if we achieve the success we are expected to, we'll be funded. It's our mission to make it so they can't pull the plug on us in 15 years."Good luck with that. The sticking-out nail is liable to get hammered down even when that nail is a whole research institute.
Tuesday, December 02, 2008
For some time now, random members of the public have been harassed, arrested and had their lives generally turned upside down by said Plod, under the guise of various ill-thought-out draconian laws that have been rushed through Parliament in recent years. And this was brushed under the carpet, or considered an acceptable price to pay to Defend our Freedoms, or some such pious nonsense.
It would of course have been even better had the victim this time been a member of zaNU-Labour rather than what passes for an Opposition these days. But both sides of the House are equally culpable for the legislation IMO - both are so desperate to be seen "doing something" that they troop obediently into the lobby even when the "something" they are voting for equates to locking up innocent people. So I don't see this as the Govt knobbling the Opposition so much as the Houses of Parliament unleashing a police state on all of us and getting caught up in it themselves.
I only wish I had a vote so I could abstain.
Wednesday, November 26, 2008
It started off with a couple of gruesome murders and another stabbing, considered to be related:
In separate but possibly related knife attacks, a former top health ministry bureaucrat and his wife were found stabbed to death in their Saitama home while the wife of another retired health ministry bureaucrat was seriously wounded in Tokyo, police said Tuesday.
All sorts of speculation ensued about terrorism or perhaps someone who is disgruntled over the pensions problems (the national pension scheme is in chaos partly because of the difficulty of reading kanji, and this is health ministry business). The police had no useful leads other than to suggest that the shoe sole of the suspect was an evil dangerous foreign Chinese invader. But most stuff here comes from China, especially if it's labelled as Japanese.
Then a few days later some guy handed himself in and claimed responsibility, saying his motivation is....
...his pet dog was put down 34 years ago, because it barked.
It wasn't even euthanased by the health ministry!
There are plenty of conspiracy theories based on a couple of embarassing flubs by the police: they announced the stabber was left-handed but this man is right handed, and the bloodied knife he presented also does not match the type they had expected. But with a confession and DNA match that is hardly going to matter.
Tuesday, November 25, 2008
As for the cost of the resulting water, at $0.30 per litre...I do not know how that compares to existing dehumidifiers but it makes a nonsense of the breathless claims that this gadget could make a useful contribution to water production. The claim of paying for itself in a couple of years is clearly absurd and refers only to an alternative of living off bottled water - and who does that?
Desalination plants can generate water at about a thousandth of the price (random google).
Monday, November 24, 2008
Well I just looked at our fuel bills. The max monthly electricity bill I can find is 440kWh (December) and the min is 142 in May (neither heating nor air-con). The summer air-con peak is about 300 and the monthly ave is comfortably below this but I will round it up a little to 10kWh per day - this is for two people and covers the vast majority of our domestic fuel use since gas is just for hot water and cooking.
The cited article (which has some silliness like conflating max rated power with actual energy usage) says that domestic energy, food, and travel make up roughly 1/3 of the total each. So by that reckoning we are using about 1/3 of the suggested max for domestic energy. Our Japanese diet is certainly lower in energy demand than a typical Western diet, being rather low in meat (and calories!). We even cycle to work most days and don't own a car. But we probably make up for it with ~2 long-haul air trips per year...
Sunday, November 23, 2008
Hmm...I wonder if I should put in a bid for the Hadley Centre...
Thursday, October 30, 2008
I don't know the details of their case, but for the background to the incident that underlies it you can look here and here. It seems like there should be a verdict early next week. I hope they have done their homework and wish them all the best.
Friday, October 24, 2008
This caught my attention in the news yesterday:
Sound familiar? Of course, it's exactly the same process that I uncovered with quick-release failures on disk brake-equipped bicycles - an underspecified bolted fastener working loose due to an applied load that exceeds the capacity of the design.Passengers at risk from track design flaw
A design flaw found in thousands of places on the country’s rail network is putting passengers at risk of a catastrophic derailment similar to the Potters Bar and Grayrigg disasters, according to a secret analysis by rail safety inspectors.
It says that the joint “may not withstand normally encountered forces imposed by the operating railway, resulting in the potential for loose fasteners and consequent changes to the configuration of the points leading to further points degradation and subsequent train derailment”.
A source close to the investigation said: “We should not just be relying on track workers to spot loose bolts in time. We need to stop the bolts from coming loose. This is about making sure that the points are fail-safe.”
Here is the full report on the Grayrigg crash. A few excerpts:
The fasteners in the third permanent way stretcher bar joint failed by unwinding. This occurs when the applied load exceeds the clamping force on the joint. The RAIB has concluded that, in this case, such loadings were a result of the passage of trains.
The RAIB has concluded that the design of the joint between the permanent way stretcher bar bracket and the switch rail at 2B points, and of other similarly configured points, was such that it could have been subjected to forces beyond its design capability and therefore the points system had significant reliance on regular inspection and maintenance to maintain safe operations.
The RAIB concludes that this incomplete understanding of the performance of S&C with non-adjustable stretcher bars, and the relationships between its design, usage, loadings, inspection and maintenance, led Network Rail to consider that the risk associated with the design was low and was being adequately controlled. This also resulted in an absence of clear and properly briefed standards for the setting up and adjustment of S&C.
[S&C - switches and crossings, a generic term for various railway track junctions]
There is lots more to wade though for anyone who is interested. But there is hardly any need. It is hardly rocket science - it took me literally seconds to find out what I needed to know via Google and this web site, once I guessed at what was going on back in 2003 (in relation to disk brakes). It is worrying to think that there are supposedly qualified engineers out there who don't know this stuff like the back of their hands...
Thursday, October 23, 2008
Actually we had several baths in the three days we were up in the mountains - that was the most spectacularly sited one though, high up in the Kurobe Gorge 5h walk from the nearest access (a little narrow-gauge rattling train). The autumn leaves were perhaps just a little short of their peak colour, but the mountain huts are about to close for the winter (in fact one of them will be completely dismantled to avoid snow damage) so we thought we had better take advantage of the good weather while it lasted.
It was pretty spectacular but a little difficult to photograph with the high contrast. And in a month or two we'll get to enjoy a second dose of the autumn colors down here in Kamakura...
Friday, October 03, 2008
Wednesday, October 01, 2008
Former Prime Minister Junichiro Koizumi on Saturday formally announced his intention to retire from politics and said he will not run in the next House of Representatives election.
Koizumi, 66, also named his second son and secretary, 27-year-old Shinjiro Koizumi, as his heir apparent to the Kanagawa No. 11 district.
Koizumi did acknowledge criticism about giving his political base to his son, saying, ‘‘I too was criticized for being a third-generation politician. However, no-one will be able to accuse my son of such a short pedigree.’’
Sunday, September 28, 2008
Thursday, September 11, 2008
"The government says its aim is to insulate every home in Britain by 2020 - and energy companies, councils and voluntary organisations will be making door-to-door visits in deprived areas to promote the scheme."Whether or not it is "enough" (and I'm sure plenty of people will be lining up to say it's inadequate), it is still a positive move. Although our old house will certainly not have cavity wall insulation by 2020, due to the fact that it has no cavity...
Back when I was living in the UK, Gordon was one of the most (if not the most) successful Chancellors ever. Now it seems that he is destined to be remembered primarily as one of the feeblest PMs. A sad but entirely predictable end to a political career (and yes I did predict it confidently, although maybe not in writing). Of course it is well-known that all political lives end in failure. I wonder why they never manage to quit while they are ahead. Did anyone ever get out at the top, leaving the masses desperate for more?
(Thinks for a minute.....ah, John Smith. That's taking things a little too far, though.)
Wednesday, September 10, 2008
The story starts way back with the IPCC AR4 drafts, actually. The very first semi-public version of this (ie the first version that people like me were allowed to see and comment on) attempted to address the issue of combining the different probabilistic estimates of climate sensitivity. They did this by averaging different pdfs that various authors have generated. It was immediately obvious to me (and possibly many others) that this was a completely invalid approach, and I said as much in my comments. In fact this episode - combined with the growing realisation that people were actually taking these outlandish "pdfs of climate sensitivity" seriously - is largely what prompted me to quickly write the "multiple constraints" paper. Anyway, this "averaging" result was removed from the next draft and I had hoped that our paper (which as far as I'm aware, no-one has argued against in the literature) might have put an end to that whole idea.
Fast forward on a few months and I saw an AGU presentation (by Reto Knutti) which appeared to again address the question of combining constraints. Its punchline was an analysis where where adding more observations appeared to result in an increase in uncertainty. Now, while it is certainly true that additional information can lead to an increase in uncertainty, such situations tend to be somewhat pathological and uncommon in practice, so I had a closer look. It only took a couple of emails back and with Reto to work out there was something badly wrong in what they had done - they had "combined" their constraints by averaging them, so I pointed out that this was invalid. Reto didn't reply to this point, and I assumed he had gone away and re-worked things. So, I was more than a little surprised that the next time I saw him speak, in the autumn of 2007, he presented the same result and mentioned that the paper (Sanderson et al) was now in press!
Time for a slight digression into why averaging the constraints is wrong. There is a standard theorem in probability theory that one always expects to learn from new information (and this somewhat woolly statement can be formalised in a rigorous manner). This general result crops up in all sorts of specific applications, and its correctness is not in doubt. Now, the sort of quadratic cost function that Sanderson et al use as their basic measure of model-data mismatch is pretty much ubiquitous in estimation, and it has a natural identification with the logarithm of the likelihood function corresponding to Gaussian uncertainties. It is no exaggeration to say that these quadratic log-likelihood cost functions underpin almost all practical probabilistic prediction and estimation in geosciences, all the way from a simple least-squares linear regression up to the advanced data assimilation methods used in modern numerical weather prediction. As a simple illustration, let's say we have an unknown variable x, and we make one observation o of it which has (Gaussian) observational uncertainty e. The resulting likelihood function P(o|x) is the Gaussian N(o,e), and taking the negative logarithm of this we reach the quadratic cost function 0.5*((x-o)/e)2. If we have more than one observation, things are little more complicated because we need to account not only for the uncertainties on each observation, but also the covariance of these uncertainties. The resulting multivariate Gaussian likelihood N(O,E) naturally gives rise to the cost function 0.5*(X-O)TE-1(X-O) where X and O are now vectors of the estimated and observed variables, and E is the covariance matrix. In the simple case where we have two observations of a single variable and the covariance of the errors E is diagonal, this simplifies to 0.5*[((x-o1)/e1)2 + ((x-o2)/e2)2] - and this is the direct sum of the cost functions that arise from taking each observation individually. And whatever the form of the covariance matrix E, the log-likelihood is never equivalent to 0.25*[((x-o1)/e1)2 + ((x-o2)/e2)2], which is the result of the "averaging the constraints" procedure that Sanderson et al invented.
The (normal, correct) cost function above corresponds to the Gaussian N((o1*e22+o2*e12)/(e12+e22), sqrt(e12*e22/(e12+e22)), this being the well-known formula for the optimal interpolation of two uncertain estimates. It is easy to see that the width of this Gaussian is lower than for either of the single-observation Gaussians, since sqrt(e12*e22/(e12+e22)) is less than the smaller of e1 and e2. (The same result holds if we use a full covariance matrix E with off-diagonal elements, but it's not as easy to show that within Blogger's limited maths capability!)
If, however, we use the average of the two single-obs cost functions, the resulting Gaussian is now N((o1*e22+o2*e12)/(e12+e22),sqrt(2*e12*e22/(e12+e22)) and it is immediate that its width, sqrt(2*e12*e22/(e12+e22), lies between e1 and e2. So, if we start with the more accurate observation, and then "combine" it with a less accurate one through this "averaging" methdology, then the final answer will always have greater uncertainty than we started with, which is contrary to the well-established theorem I mentioned previously. Effectively, this method is asserting that a weak observation with wide uncertainty should cause us to forget what we already knew!
So in summary, averaging constraints is obviously nonsense that has no possible probabilistic interpretation. I spent a lot of time late last year trying to explain this in simple terms to several of the authors of the Sanderson et al paper, and in response they gave every appearance of trying to argue that averaging was a valid approach, although they also accused me of reading too much into the paper. To be fair, they had not directly generated any posterior pdfs, and did not explicitly state that these cost functions were log-likelihoods, but given (a) the theoretical background I have briefly outlined above, and (b) the supposed increase in uncertainty as more observations were considered had been repeatedly presented as the main result of this work, I don't think my reading of it was unreasonable. Moreover, if they had really meant to disavow this interpretation, then it is puzzling that they continued to defend the methodology as valid. Myles Allen even challenged me to write a comment if I thought their paper was wrong, and given the potential impact of this erroneous "averaging the constraints" meme if it is widely adopted, I was originally tempted to do so. But I have just looked for the paper on-line, and am surprised (but pleased) to see several changes to the paper that have been made subsequent to our discussions. For starters, they have removed the most contentious claim - which was previously made in the abstract as well as the main text - that their cost function provides a constraint on climate sensitivity that weakens as more observations are used. In fact the paper now contains an entirely new paragraph that was not there before. This explicitly states that their results cannot be interpreted in terms of log-likelihoods. They could have gone a little further and admitted that the widths of their cost functions cannot be directly related to uncertainty in any way whatsoever, but I suppose that would have been rather embarrassing given that the explicit goal of the paper is to explore this issue. They have also added a little discussion of their reasoning behind the averaging method (which was not justified at all in the earlier version of the paper). Now it is explicitly presented as alternative to the "incorrect" alternative of adding the individual terms, which (as they state) is wrong if uncertainties are not independent. It is disappointing that they didn't find the space to mention here that the obvious alternative to adding the terms, if they are not independent, is just to use the full covariance matrix as I have outlined above. For while of course the addition of individual constraints is incorrect unless the uncertainties are judged to be independent, this may at least be a reasonable approximation if the dependencies are small (and it could in principle be biased either way), whereas their curious "averaging" method is guaranteed to be wrong for all possible covariance structures, with the resulting uncertainty guaranteed to be too wide. Indeed it is trivial to observe that averaging is algebraically equivalent to addition, but for the arbitrary and unjustifiable multiplicative scaling factor of sqrt(N) on the width of the implied joint likelihood function, where N is the number of individual terms.
I guess the authors would claim that these last-minute changes to the paper are nothing more than cosmetic - just a matter of explaining things in simple enough terms for people like me to understand, rather than representing any substantive modifications to their previous claims. (They certainly don't acknowledge my help in fixing the problems in the original version.) The real proof of the pudding will be in whether they and others continue to use this "averaging the constraints" meme...I will watch with interest to see if and how it spreads...
Wednesday, September 03, 2008
Snow fell on Mt Fuji Aug 9 at earliest time ever
However, it seems that this new record is due to an arbitrary decision to redefine some hail as snow. It has still been unseasonally wet and cold for the latter part of the summer - which I would have been quite happy about but for the fact that we were hoping to have another trip into the mountains. I'm certainly glad we made the effort earlier.
Police in Japan have been left red-faced by an apparent murder that turned out to be an unusual case of mistaken identity.
Back at the local police headquarters, officials notified reporters who had turned up early the same morning to cover an annual earthquake drill. They began preparing to write up the launch of a major murder investigation.
Dozens of extra officers were dispatched to interview potential witnesses, while the evening edition of the local newspaper carried a report of the gruesome find, complete with a photograph of the body's resting place.
What they don't mention in this story is that by the time the police realised it was a doll, it had already confessed to two charges of shoplifting and one of child neglect.
It is now rumoured to be considering standing as a candidate in the forthcoming election for the new Prime Minister.
So he's up to 8 out of 14, which means the p-value is still below 5%, but it is heading in the right direction. There is no sign of the September forecast yet. The August one only appeared on the 8th, by which time it was already well set to be very a wet month with cold days, but of course he claims that this forecast was really made some time previously...
Tuesday, August 26, 2008
Convention on the Elimination of All Forms of Racial Discrimination in 1998 if it is still inconceivable 10 years later that they should actually take the promised and required action to eliminate and prohibit racial discrimination within their jurisdiction.
Friday, August 15, 2008
Apparently the Japanese tradition of fireworks displays is just for entertaintment purposes with no more specific significance (unlike the UK). But there are worse ways of spending a summer evening than sitting on a beach watching the display...especially as this year was fine and clear, with a gentle breeze clearing the smoke (towards the far end of the beach). Last time we went, a few years ago, the rockets all disappeared into the smoggy cloud. Often we are up in the mountains at this time of year, which is the best place to be now it is hot and sweaty every day.
Not a very exciting video but here it is. I really just wanted an excuse to play with the new iMovie application.
Monday, August 04, 2008
Sunday, August 03, 2008
Saturday, August 02, 2008
Of course butter is not a major staple foodstuff in Japan - indeed "butter-smelling" is a (rather rare) insult to westerners, and I sense a bit of smug satisfaction with a Japanese Agriculture Ministry official saying “Personally, I can happily switch to margarine.” But even so, it seems amazing to me that a developed country can simply run out of a basic foodstuff that is widely used and was readily available until recently. It is fabulously expensive, too, at something like ¥600 for a 200g stick, which is about 4 times the price in the UK. You'd think at that price someone could have managed to import some...but of course protecting the interests of the farming lobby is a higher priority than providing food for the population, so the import tariff is 360% (apparently).
So now he is up to 6 correct forecasts out of 12 so far this year (temp and precip for 6 months, June being curiously absent). That still represents a p-value of less than 2% based on his claimed 80% accuracy. That is, if his forecasts really had a long-run probability of 80% of validating, then the probability of getting 6 or fewer correct out of 12 is under 2%. On the other hand, it is better than chance, given that his ranges generally cover less than 50% of a reasonable climatological range. (That doesn't mean much, given that he only publishes the forecast a day or two into the month. The very heavy rain at the start of July was easily predicted by everyone at that point. Also, there is the question as to why June's forecast was not released...but there is little point in speculating on that point.)
Tuesday, July 29, 2008
Wednesday, July 23, 2008
5.7 Views and facts must not be misrepresented. Views must also be presented with due weight over appropriate timeframes.Now I'd say it is basically indisputable that Durkin's Swindle fails repeatedly on that score. There is plenty of documentation here.
But OFCOM basically decided that Section 5.7 did not apply, and instead treated the complaint as if it was made under Section 2.2, where in order for a breach to have occurred, the programme would have to actually directly cause harm! Why did they do this?
OFCOM'S bizarre claim is that climate science is not covered by the rubric: major matters relating to current public policy. Tell that to the G8 ministers...even if the fact of anthropogenic climate change is considered indisputable, the magnitude and speed of future changes is a major issue that has a big impact on rational policy decisions (with substantial uncertainty even within the mainstream scientific consensus).
In order to justify their judgement, OFCOM even present an open invitation to HIV denialists, effectively stating that they would be allowed to broadcast their theories outside the scope of the Broadcasting Code. No doubt some IDiots will also have pricked up their ears at this news. What a great day for British Broadcasting!
I hope that Rado et al appeal successfully against this cowardly evasion.
(I'm off on holiday for a few days, so any silly flames and trolling will stay up for a little while. But they will be deleted on my return.)
Tuesday, July 22, 2008
Although C4 did get criticised for the unfair way it had treated Sir David King, Prof Carl Wunsch and the IPCC (and will have to publish an apology), they got off remarkably lightly on the question of "materially misleading", due to the way that OFCOM wriggled their way out of making any meaningful judgement. Because it is a "factual" programme rather than merely "news", there is no requirement of "due accuracy" on the broadcaster. The appropriate test for a breach of the code is a rather higher one, that the program must actually mislead "so as to cause harm or offence". Since the science was already settled, no-one will have believed the Swindle, and no policy decisions will be made on it, therefore there is no chance of it materially causing harm. So in summary it seems that (1) OFCOM doesn't care about the factual accuracy precisely because the program is supposedly "factual", and (2) climate science is not "a matter of political or industrial controversy or matter relating to current public policy", thus OFCOM doesn't require the presentation of it to be impartial or balanced (the economics-focussed section of the program did breach this bit of the code, however). It sort of has a valid internal logic in a smugly complacent middle-class sort of way, but leaves me wondering what OFCOM is actually for.
OTOH, in OFCOM's favour, I've not seen any evidence that the Swindle actually did help to convince anyone of anything (other than Durkin's dishonesty).
Monday, July 21, 2008
As for the content, Tim Lambert has already fisked it briefly (see also here and here). The journalist in question is well known to be a clueless twit, although it has been rumoured that his behaviour is actually an elaborate hoax designed to undermine support for the peerage.
Saturday, July 19, 2008
Despite criticising Durkin for how the IPCC and Sir David King were treated, OFCOM seems to agree with my judgement that Wunsch was not actually badly misrepresented (although I can of course understand him being unhappy about how he was presented). And if Singer had had the sense to correctly attribute Lovelock's comments on "breeding pairs" etc (or report more accurately what David King had actually said) that would have taken the sting out of another of the major complaints, without changing the message substantially.
So it doesn't seem like the complaint has actually resulted in that definitive an outcome, although will have to wait to next week for the official version (and I suspect science correspondents may be harsher on Durkin than the Guardian's media correspondent was). I do hope that scientists who may in the future be tempted to appear in anything related to Martin Durkin and/or his company WAG TV will do a quick google search to find out what a piece of work he is, though.
Wednesday, July 16, 2008
Incidentally, I heard recently that someone here who was due to retire soon has decided (or been persuaded) to stay on for a few more years. Several staff members have died since I've been here, but off the top of my head, I can't think of anyone who has actually retired normally. [In contrast, back in the UK, it seemed rare to actually make it through to retirement age - many more people in my lab had early retirements or voluntary redundancy than a standard retirement at the appropriate age.]
Friday, July 04, 2008
Anyway, on with July. It's going to be extremely wet ("probably one of the three wettest Julys on record") with a rainfall of 160-250% of the average over England and Wales. Just to be difficult, his temperature prediction is for the daily max rather than mean as previously. This is predicted to be 0.2-1C below mean, and he also says that the mins will probably not be below normal. I reckon I can therefore reasonably assign a range of about -0.5C to +0.2C for the means (for comparison with data). If the mean is outside this range, then it is hard to see the max being in Corbyn's prediction. I think this is probably generous to him, but will be happy to hear alternative suggestions.
As well as the UK weather, he has thrown in some typhoon predictions for free. Apparently there's a 60% chance of Japan taking a hit in the first half of the month or so. No sign of them so far and it has been particularly cold here this year, although I don't know what the sea surface temperature is looking like in the formation zone.
I've been away, BTW, for those who were wondering if I'd given up entirely.
Friday, June 13, 2008
Tuesday, June 10, 2008
Do not panic-buy petrol, PM tells motorists ahead of fuel strike.
We enjoyed the Fool Protests and the farcical "Dump the pump (but only one day a week because we don't really want to buy less fuel but just want to cause some inconvenience to others)" campaign several years ago before moving to Japan. The roads were noticeably quieter while it lasted. But all good things must come to an end, and it did not last long. (Addicts of cheap unlimited oil please note.)
As for the poor striking drivers...my heart bleeds. Actually I support their right to strike if they feel like it, but I also see that their average salary (if that £39,000 quote is accurate) is well in excess of a typical scientist. Can you hear captains of industry and academia bleating about how we need to train more lorry-drivers? I thought not.
Thursday, June 05, 2008
The politician in question is the Japanese justic minister, and the topic is the recording of interrogation sessions by the Japanese police in order that the courts can judge how "voluntary" the resulting confessions are. There is increasing pressure for this, as the public slowly wakes up to the standard operating practice of the judicial system here (such as 3 weeks in custody before any charges are laid, during which time the victim is treated to sleep deprivation and coercive interrogations aiming at a "confession" which - contrary to the letter of the law - is itself sufficient for a conviction even if subsequently withdrawn).
He is obviously worried by the plummeting conviction rate - down to 97.1% at the last count. This politician is the same moron who recently said he is in favour of the death penalty because the Japanese place much more importance on the value of life than the barbarian Westerners and that didn't get him laughed out of a job either. For that matter, neither did his boast that his friend knew a member of Al-Quaida who regularly visited Japan, and that he had advance knowledge of the Bali bombing but did nothing about it. [That latter story was easily debunked as the delusions of a fantasist, BTW.]
Monday, June 02, 2008
Steve McIntyre was quick to present a hypothetical new surface temperature record, based on slowly phasing out buckets over the 2nd half of the century.
This gives a greatly reduced warming trend over the last 50 and even 30 years, which got Roger Pielke very excited. However, even though Steve may be right about the buckets themselves, his analysis (which, to be fair, he did not present as anything authoritative) ignores the fact that SST observations have increasingly come from other sources such as satellites and buoys. (Furthermore, that graph above seems to show a change of 0.3C for the global mean temperature, which is hard to justify since the adjustment under discussion is only 0.3C for the SST measurements made by some ships, albeit most of them.) I happen to have used some satellite SST data in previous research, and on checking the details I see that the Pathfinder AVHRR were flying from late 1981 (there may even have been older missions for all I know, but I would guess not). This fleet of satellites has provided hi-res global coverage on a regular basis from 1985 (and probably something from the few years prior), so although I do not know the details of the global SST analysis it seems inconceivable that bucket measurements from ships played much of a part subsequent to that date. Turning to buoys, Wikipedia tells me that the US National Data Buoy Development Program started in 1967 and the National Data Buoy Center was formed in 1970, so it seems that at least some data was coming from this source back then too (and note that even if the volume of data was quite small, its relative accuracy may make it outweigh a lot of ship measurements).
So I don't know exactly what the charges will be, but I suspect that the graphic here attributed to CRU is a reasonable first guess.
As you should be able to see at a glance (although Roger apparently cannot), the maximum change in the smoothed record shown there is a little short of 0.2C globally, which is just as expected given a maximum change of no more than 0.3C for the ocean (70% of the earth's surface).
So how does this affect the IPCC's latest report? Well, in the Technical Summary, there is a nice figure (Figure TS.6) which gives 25, 50, 100 and 150 year trends. So let's give this a little update shall we?
Actually, I cannot reproduce either that graph or the Independent's one precisely, as I do not know exactly how they did the smoothing, or (in the case of the IPCC) which data set they used. But this seems acceptably close to both (click for larger version):
The original data are the crosses and the blue line (5 year boxcar smooth), with the new smoothed data forming the dotted blue line which trends down more slowly from 1945-1960. The adjustment I used was a linear term which falls from 70%*0.3C in 1946, to 0 in 1960. Obviously I don't claim this is authoritative but I do believe it is more plausible than a number of other graphs I have seen on the intertubes...
The solid straight lines are the trends plotted by the IPCC. The dotted ones are equivalent trends of the adjusted data. With the help of a bit of rounding, the 50 year trend (1956-2005) changes from 0.13 to 0.12C/decade. By cherry-picking the start date to be 1946 I can get a change as large as 20%, from 0.11C/decade to 0.09C/decade. Woohoo. Let the blogorrhea continue...
Update. It certainly looks like the Independent graphic attributed to CRU is smoothed over rather more than 5 years. Roger says it is a 21y binomial filter, and if that is correct then the adjustment presented in that graph there must be just a hand-drawn guess rather than the results of any realistic calculation, since a 21y smooth would smudge out any adjustment over a rather longer interval than indicated. It's even possible that the attribution to CRU is just for the original data, and the adjustment was drawn on by a journalist. So don't take any of this too seriously for now. I would guess that the data analysts may take this opportunity to have another look at the various assumptions underlying the splicing together of different data sources, and by the time they are done there may be a whole bunch of minor adjustments throughout the time series.
Sunday, June 01, 2008
So now Corbyn is down to a 40% success rate for the year so far (4 hits from 10 forecasts). Even if he gets every remaining month for the rest of the year perfect for temperature and rainfall, he cannot climb back up to his claimed 80% success rate.
Friday, May 30, 2008
Thursday, May 29, 2008
It seems like it won't make much difference to climate predictions, although maybe one should expect it to reduce our estimates of both aerosol cooling and climate sensitivity marginally (I haven't read that linked commentary yet, so don't know how much detail they go into). It will also make it easier for the models to simulate the observed climate history. In fact one could almost portray this as another victory for modelling over observations, since the models have always struggled to reproduce this rather surprising dip in temperatures (eg SPM Fig 4). I've asked people about this problem myself in various seminars, and never got much of an answer. It's pretty shocking that such a problem could have been overlooked for so long.
It wasn't overlooked by everyone, actually. But I anticipate that plenty of people will try their best to avoid looking and linking in that particular direction...
Sunday, May 25, 2008
I wasn't going to bother blogging this, as there is really not that much to be said that has not already been covered at length. But I sent it to someone for comments, and (along with replying) he sent it to a bunch of other people, so there is little point in trying to pretend it doesn't exist.
It's a re-writing of the uniform-prior-doesn't-work stuff, of course. Although I had given up on trying to get that published some time ago, the topic still seems to have plenty of relevance, and no-one else has written anything about it in the meantime. I also have a 500 quid bet to win with jules over its next rejection. So we decided to warm it over and try again. The moderately new angle this time is to add some simple economic analysis to show that these things really matter. In principle it is obvious that by changing the prior, we change the posterior and this will change results of an economic calculation, but I was a little surprised to find that swapping between U[0,20C] and U[0,10C] (both of which priors have been used in the literature, even by the same authors in consecutive papers) can change the expected cost of climate change by a factor of more than 2!
We have also gone much further than before in looking at the robustness of results when any attempt at a reasonable prior is chosen. This was one of the more useful criticisms raised over the last iteration, and we no longer have the space limitations of previous manuscripts. The conclusion seems clear - one cannot generate these silly pdfs which assign high probability to very high sensitivity, other than by starting with strong (IMO ridiculous) prior belief in high sensitivity, and then ignoring almost all evidence to the contrary. Whether or not such a statement is publishable or not (at least, publishable by us), remains to be seen. I'm not exactly holding my breath, but would be very happy to have my pessimism proved wrong.
Monday, May 19, 2008
By the normal definition, this simple null hypothesis significance test should reject roughly 5% of the models. Eg Wilks Ch 5 again "the null hypothesis is rejected if the probability (as represented by the null distribution) of the test statistic, and all other results at least as unfavourable to the null hypothesis, is less than or equal to the test level. [...] Commonly the 5% level is chosen" (which we are using here).
I asked Roger what range of observed trends would pass his consistency test and he replied with -0.05 to 0.45C/decade. I then asked Roger how many of the models would pass his proposed test, and instead of answering, he ducked the question, sarcastically accusing me of a "nice switch" because I asked him about the finite sample rather than the fitted Gaussian. They are the same thing, Roger (to within sampling error). Indeed, the Gaussian was specifically constructed to agree with the sample data (in mean and variance, and it visually matches the whole shape pretty well).
The answer that Roger refused to provide is that 13/55 = 24% of the discrete sample lies outside his "consistent with the models at the 95% level" interval (from the graph, you can read off directly that it is at least 10, which is 18% of the sample, and at most 19, or 35%).
But that's only with my sneaky dishonest switch to the finite sample of models. If we use the fitted Gaussian instead, then roughly 23% of it lies outside Rogers proposed "consistent at the 95% level" interval. So that's entirely different from the 24% of models that are outside his range, and supports his claims...
I guess if you squint and wave your hands, 23% is pretty close to 5%. Close enough to justify a sequence of patronising posts accusing me and RC of being wrong, and all climate scientists of politicising climate science and trying to shut down the debate, anyway. These damn data and their pro-global-warming agenda. Life would be so much easier if we didn't have to do all these difficult sums.
I'm actually quite enjoying the debate over whether the temperature trend is inconsistent with the models at the "Pielke 5%" level :-) And so, apparently, are a large number of readers.
Sunday, May 18, 2008
The models provided a distribution of predictions about the real-world trend over the 8 years 2000-2007 inclusive. However, we have only one realisation of the real-world trend, even though there are various observational analyses of it. The spread of observational analyses is dependent on observational error and their distribution is (one hopes) roughly centred on the specific instance of the true temperature trend over that one interval, whereas the spread of forecasts depends on the (much larger) natural variability of the system and this distribution is centred on the models' estimate of the underlying forced response. Of course these distributions aren't the same, even in mean let alone width. There is no way they could possibly be expected to be the same (excepting some implausible coincidences). So of course when Roger asks Megan if these distributions differ, it is easy to see that they do. But what is that supposed to show?
People tend to get unreasonably hot under the collar in discussions about climate science, so let's change the scenario to a less charged situation. Roger, please riddle me this:
I have an apple sitting in front of me, mass unknown. I
1. If I weight the apple once, what range of observed weights X are consistent with my estimate of 100±50g?
2. If I weigh the apple 100 times with 100 different sets of scales (each set of scales having independent errors of the same magnitude), what range of observed weight distributions are consistent with my estimate for the apple's mass of 100±50g. Hint: the distribution of observed weights can be approximated by the Gaussian form X±5g for some X. I am asking what values for X, the mean of the set of observations, would be consistent with (at the 95% level) my estimate for the true mass.
You can also ask Megan for help, if you like - but if so, please show her my exact words rather than trying to "interpret" them for her as you "interpreted" the question about climate models and observations. You can reassure her that I'm not looking for precise answers to N decimal places to a tricky mathematical problem so much as a understanding of the conceptual difference between the uncertainty in a prediction, and the uncertainty in the measurement of a single instance. It is not a trick question, merely a trivial one.
Or, dressing up the same issue in another format:
If the weather forecast for today says that the temperature should be 20±1C, and the thermometer in my garden says 19.4±0.1C, then I hope we would all agree that the observation is consistent with the forecast. Would that conclusion change if I had 10 thermometers, half of which said 19.4±0.1C and half 19.5±0.1C? Of course, in this case the distribution of observations is clearly seen to be markedly different from the distribution of the forecast. Nevertheless, the true temperature is just as predicted (within the forecast uncertainty). If there is anyone (not just Roger) who thinks that the mean observation of 19.45C is inconsistent with the forecast, please let me know what range of observed temperatures would be consistent.
Friday, May 16, 2008
Unfortunately, the bit where he tries cherry picking a shorter interval Jan 2001 - Mar 2008 and claims "there is a strong argument to be made that these distributions are inconsistent with one another" is just as wrong as the nonsense he came up with previously.
Really, I would have thought that if my previous post wasn't clear enough, he could have consulted a numerate undergraduate to explain it to him (or simply asked me about it) rather than just repeating the same stupid errors over and over and over and over again. This isn't politics where you can create your own reality, Roger.
So let's look at the interval Jan 2001-Mar 2008. I say (or rather, IDL's linfit procedure says) the trend for these monthly data from HadCRU is -0.1C/decade, which seems to agree with the value on Roger's graph.
The null distribution over this shorter interval of 7.25 years will be a little broader than the N(0.19,0.21) that I used previously, for exactly the same reason that the 20-year trend distribution is much tighter than the 8-year distribution (averaging out of short-term variability). I can't be bothered trying to calculate it from the data, but N(0.19,0.23) should be a reasonable estimate (based on an assumption of white noise spectrum, which isn't precisely correct but won't be horribly wrong). This adjustment doesn't actually matter for the overall conclusion, but it is important to be aware of it if Roger starts to cherry-pick even shorter intervals.
So, where does -0.1 lie in the null distribution? About 1.26 standard deviations from the mean, well within the 95% interval (which is numerically (-0.27,0.65) in this case). Even if the null hypothesis was true, there would be about a 21% probability of observing data this "extreme". There's nothing remotely unusual about it.
So no, Roger, you are wrong again.
Thursday, May 15, 2008
Roger Pielke has been saying some truly bizarre and nonsensical things recently. The pick of the bunch IMO is this post. The underlying question is: Are the models consistent with the observations over the last 8 years?
So Roger takes the ensemble of model outputs (8 year trend as analysed in this RC post), and then plots some observational estimates (about which more later), which clearly lie well inside the 95% range of the model predictions, and apparently without any shame or embarrassment adds the obviously wrong statement:
"one would conclude that UKMET, RSS, and UAH are inconsistent with the models".Um....no one would not:
I challenged this obvious absurdity and repeatedly asked him to back it up with a calculation. After a lot of ducking and weaving, about the 30th comment under the post, he eventually admits "I honestly don't know what the proper test is". Isn't thinking about the proper test a prerequisite for confidently asserting that the models fail it? Anyway, I'll walk through it here very slowly for the hard of understanding. I'll use Wilks "Statistical methods in the atmospheric sciences" (I have the 1st edition), and in particular Chapter 5: "Hypothesis testing". It opens:
Formal testing of hypotheses, also know as significance testing, is generally covered extensively in introductory courses in statistics. Accordingly, this chapter will review only the basic concepts behind formal hypothesis tests...[cut]
and then continues with:
5.1.3 The elements of any hypothesis test
Any hypothesis test proceeds according to the following five steps:
1. Identify a test statistic that is appropriate to the data and question at hand.
This is a gimme. Obviously, the question that Roger has posed is about the 8-year trend of observed mean surface temperature. I'm going to use an ordinary least squares (OLS) fit because that is what is already available for the models, and it is also by far the most commonly used method for trend estimation and has well understood properties. For some unstated reason, Roger chose to use Cochrane-Orcutt estimates for the observed data that he plotted in his picture, but I do not know how well that method performs for such a short time series or how it compares to OLS. Anyone who wishes to repeat the analysis using C-O should find it easy enough in principle, they will need to get the raw model output (freely available) and analyse it in that manner. I would bet a large sum of money that this will not change the results qualitatively.
2. Define a null hypothesis.
Easy enough, the null hypothesis H0 here that I wish to test is that the models correctly predict the planetary temperature trend over 2000-2007. If anyone has any other suggestion for what null hypothesis makes sense in this situation, I'm all ears.
3. Define an alternative hypothesis.
"H0 is false". This all seems too easy so far....there must be something scary around the corner.
4. Obtain the null distribution, which is simply the sampling distribution of the test statistic given that the null hypothesis is true.
OK, now the real work - such as it is - starts. First, we have the distribution of trends predicted by the models. As RC have shown, this is well approximated by a Gaussian N(0.19,0.21). (I am going to stick with decadal trends throughout rather than using a mix of time scales to give me less chance of embarassingly dropping factors of 10 as Roger has done in several places in his post. He has also plotted his blue "95%" lines in the wrong place too, but I've got bigger fish to fry.) There are firm theoretical reasons why we should expect a Gaussian to provide a good fit (basically the Central Limit Theorem). This distribution isn't quite what we need, however. The model output (as analysed) uses perfect knowledge of the model temperature, whereas the observed estimate for the planet is calculated from limited observational coverage. In fact, CRU estimate their observational errors at about 0.025 for each year's mean (at one standard deviation). This introduces a small additional uncertainty of about 0.04 on the decadal trend. That is, if the true planetary trend is X, say, then the observational analysis will give us a number in the range [X-0.08,X+0.08] with 95% probability.
Putting that together with the model output, we get the result that if the null hypothesis is true and the models' prediction of N(0.19,0.21) for the true planetary trend is correct, then the sampling distribution for the observed trend should also be N(0.19,0.21). I calculated 0.21 for the standard deviation there by adding the two uncertainties of 0.21 and 0.04 in quadrature (ie squaring, adding, taking the square root). This is the correct formula under the assumption that the observational error is independent of the true planetary temperature, which seems natural enough.
So, as I had guessed in my comments to Roger's post, considering observational uncertainty here has a negligible effect (is rounded off completely), so we could have simply used the existing model spread as the null distribution. Using this approach generally makes such tests stiffer than they should be, but it is often a small effect.
5. Compare the observed test statistic to the null distribution. If the test statistic falls in a sufficiently improbable region of the null distribution, H0 is rejected as too unlikely to have been true given the observed evidence. If the test statistic falls within the range of "ordinary" values described by the null distribution, the test statistic is seen as consistent with H0 which is then not rejected. [my emphasis]
OK, let's have a look at the test statistic. For HADCRU, the least squares trend is....0.11C/decade. That is a simple least squares to the last 8 full year values of these data. (I generally use the variance-adjusted version, on the ground that if they think there is a reason to adjust the variance, I see no reason to presume that this harms their analysis. It doesn't affect the conclusions of course.)
So, where does 0.11 lie in the null distribution N(0.19,0.21)? Just about slap bang in the middle, that's where. OK, it is marginally lower than the mean (by a whole 0.38 standard deviations), but actually closer to the mean than one could generally hope for, even if the null is true. In fact the probability of a sample statistic from the null distribution being worse than the observed test statistic is a whopping 70% (this value being 1 minus the integral of a Gaussian from -0.38 to +0.38 standard deviations)!
So what do we conclude from this?
First, that the data are obviously not inconsistent with the models at the 5% level.
Second...well I leave readers to draw their own conclusions about Roger "I honestly don't know what the proper test is" Pielke.