readings in computer science found in a time capsule

recently, I was re-reading the classic old article on Smashing the stack for fun and profit in phrack, and wondering where the positive alternative lesson might be. Quite a while ago, I did a port of the SR(Synchronized Resources) programming language out of Arizona, to the newly minted RS6000 system out of IBM. The language is an elegant system for teaching principles of concurrency in lots of nice ways - at the time we didnt have a single agreed way to do it (e.g. wrong like in C11, or possibly ok in Java), so there was a need for a pedagogic approach and SR together with a nice book was very cool. However, we'd just bought a bunch of new IBM AIX systems with the new POWER PC RISC processor, which had a whole new instruction set - lets leave aside the amising idea of a "reduced" or "regular" instruction set that includes "floating point multiple and add" in its portfolio. However, in terms of registers, its a pretty nice system.

So why is porting SR to the Power CPU reminiscent of stack smashing, I hear you cry?
Because, you need to implement the threads system it uses to emulate real multi-core, I hear myself answer. And what does it take to do that, you continue? well you need to save the current context (like all the registers and thread state (i.e. PC, stack) move to a different thread/stack by calling the scheduler with a pointer to that context. To do this involves becoming familiar with the stack format used for most programming languages.as well as all the important (i.e. all) registers etc. So basically, slightly more than the Phrack folks....basically, writing stuff that moves to another exection context, but can get back again correctly, is harder:-)
You can see some nice examples of the context switch stuff for a variety of processors in the (not supported) SR archive

Meanwhile, I was also reading about the integrity check used for DNSSEC caches. so that will be reported elsewhere, but the interesting thing is that its a weak version of the IP header (and TCP) checksum algorithm. Again this is something that excercises computer science #101 - you need to add up all the 16 bit fields in the buffer (imagine you have a n byte buffer, then if n is odd, add a zero value byte and sum it as a vector of 16 bit values using "end round carry" (or ones-complement) artithmentic - basically, you have a 32 bit accumulator, and loop over the buffer. when you are done, you do one more thing - in the checksum case: check any bits above the 16 least sig are set, and fold them in (add again) and then do one more add in case that overflows too. In the integrity check case, just 0 any bits in the more significant 16 bits (i.e. && accumulator with 0x0ffff). For the ARM IP cksum case, see this
code with inline asm - the crucial bit's lines 78&79 after the bne loop.

amusing, back in the early days, i remember someone loop unrolling asm code for hte M68000 (that's not a 68020 or 68010, but 68000 - that was sold as a "Codata" computer in the uk, but was a Sun-1, i think, never sold by Sun) - the code went slower as the loop was now to big to fit in the miniscule instruction cache of said CPU...

hum...........what could possibly go oddly wrong with the allegedly simpler (by 1 instruction) integrity check algorithm? I leave that as an exercise for the coder

counciling the UK research councils

The UK research councils are unique in the way proposals are reviewed in several ways (in my experience)

firstly, unlike almost any other research funding agency (unlike DARPA, NSF in USA, CNRS in france, DFG in germany, the Japanese, scandinavian, and just about any other national funding agency I have dine reviewing for, which is a lot), the officers are not seconded experts so the assignment of reviewers depends on the reviewers self description - notoriously inaccurate. Unlike a journal (where the editor is an expert) there's not really a _peer_ review assignment process

secondly, the reviews can be rebutted by the proposer, but since the reviewers don't see each others reviews, they can't be calibrated against each other (unlike a conference or journal)

thirdly, the panel are not the reviewers and are not allowed to re-review the proposal, even if they are experts and only in exceptional circumstances will they discount an obviously incompetent or inappropriate review.

As a recipient of significant funding from the research councils, I am not expressing this through some sour grapes emotion, but more on behalf of my bewildered junior colleagues, who frequently receive inexplicably odd reviews and panel decisions. This is not good for community trust in the system - it may be ok at the obvious top 5-10% of proposals, but it results effectively in random decisions quite shortly after the very top (all 6s) ranked research. This is not good for confidence.

I can't give examples as that would be a breach of confidentiality, but everyone I know can tell a tale.

Maybe the new system after the recent review will involve people who have an answer to why the EPSRC and other councils should have a unique, and uniquely odd system. I have never heard an evidence based response to the comments above, which I have made several times to officials from the research councils. of course, since they themselves are not seconded from the community, how would they know, in any case. However, they could try talking to colleagues in other countries a bit more and see what works (or not) to persuade us that this is not just some random "we do it this way because we always have done"...

Part II Principles of Communications, 2015, to Dec 2

Wrapped up Traffic Management, including
idea of signaling protocols, and
space/time scales of different techniques;
and review of the course contents.

Planning to do revision classes next term.
Supervision notes will be linked to course web page after end of term.

[systems structures lecture moved to background]

some notes/problems from supervisions added
now

Part II Principles of Communications, 2015, to Nov 27

This week is

• Shared Media
• Mesh Network Capacity
• Coding and Multihop Radio (Cope)

Two useful network coding primer and coded storage articles...
and just what is a shim?

Part II Principles of Communications, 2015, to Nov 20

this week, covered scheduling, work conservation, max/min fairness, and PGPS approximations + queue jump!

Part II Principles of Communications, 2015, to Nov 13

Optimization (Nov 9) - background paper:-

from Princeton group

TCP in the wild - Optical Carrier Speed
table + Reminder about R-squared

Schedulers&Rounds....up to WFQ.

Part II Principles of Communications, 2015, to Nov 6

Flow Control, Congestion Control,
open and closed loop systems

Control Theory -
time domain model
transform to frequency domain
recipies for laplace transforms
stability & efficiency
proportional, integral and differential (and PID) controllers.

Online slides (PPT and PDF) are ok -
apologies for symbol/font problem in some of the printed pages!

Part II Principles of Communications, 2015, to Oct 30

Previously, Compact routing, Central routing (fibbing to link state),&path vector

This week covered
interdomain routing, including AS paths and deadlocks,
for which see interdomain revision notes/slides
+
multicast,
+
random routing

Just started on error and flow control.
[As usual, wikipedia is great on gaussian elimination ]

Pretty much as per schedule

Part II Principles of Communications, 2015, to Oct 23

This week has been about inter-domain routing, and BGP.
Covered background business relationships (peering, customer/provider etc), the protocol, the attributes/selection, SPP, and discussed scaling, convergence, and problems (like bad gadgets, wedgies) - as always, data is out of date, but Geoff Huston to the rescue...

Part II Principles of Communications, 2015

This week (to Dec 16) I should have covered the graph material (random graphs + alpha&beta models of small world/clustered graphs), and
Compact and Centralised routing - viz
http://www.cl.cam.ac.uk/teaching/1516/PrincComm/slides/schedule-2015.html

On graphs and networks there's a new really nice book by Jon Kleinberg


There's some lack of precision about the terms "small world" but basically, (wikipedia is your friend) a scale-free network refers to the power law degree distribution, and consequential small diameter (and some clustering), which leads to the small world property. Not all small world networks are scale free, but scale free networks are small world...

The Smell of Memory

scientists in cambridge have recently figured out how to code the smell of memory, not just a smell of one thing that evokes a particular memory, but the underlying phenomenon that is memory itself. It turns out that it is not so abstract, and that there are only really 7 key parameters (somewhat like taste, which is, of course, related to smell in any case). Working backwards from the examples of smells that evoke memories, using a statistical technique called PCA, scientists can now code the entire space that is memory - so not only evoking a particular memory, but replaying everything at once.

Of course, there are severe dangers of synaesthesia with this technique, so it will only be allowed in key, socially beneficial situations, such as court cases, or TV interviews with politicians.

And it must be noted that there will inevitably be people who have an innate fear of smell - loosely related to the smell of fear, but far more disabling in this context, except where you'd like the right to be forgotten - de-oderant amnesia.

Re-use ill-considered (twitter and hypothermia)

I've been reading a lot about ethics recently for a forthcoming workshop in Oxford on the topic.
It seems that a lot of the book keeping elements of ethics have been defined (I guess, perhaps, in reaction to the lamentably long list of unethical things that are done by all and sundry in online media commercial appropriation of the essence of you, and dodgy medical research re-owning folk knowledge on drugs from rainforest eco-systems, or testing stuff on people who have no political or economic choices...

So there's a long list of stuff we (as researchers) have to go through.....

but it seems to me that quite near the top of the list ought to be something about re-use

In computing (and in general in engineering), re-use of stuff (re-cycling harware, and re-cycling good ideas, or software) is regarded as a good thing - rep-purposing too - the internet came out of dual-use (re-purposing a survivable defense network, as a cheap extnesive global communications utility for the public)...

but the real elephant in the room for me is re-use of stuff in a new context without ethical consideration.

Let me give two examples (from the title of this post)

1/ someone (NOT ill-advisedly) tweets that, unless the snow in an airport is cleared soon, he will blow it up. This is a daft joke which maybe to his friends is funny coz he's that sort of a guy.
That's not the problem. THe problem is that it got re-tweeted or forward to the police - whoever did that, did they stop to consider the original intent. Or was this thoughtless? or possibly even malicious? why did the police not regard this as potentially a "crank call"? were they not wasting police time

2/ research on hypothermia received a boost from the Nazi doctors trying stuff out on people in concentration camps. The research actually turned out to be useful. Should we deny ourselves the benefit, despite the awful provenance of the results?

Some of this can be reduced (in a classic reductionist manner) by looking at cost-benefit tradeoffs. What are the risks the tweet really is a bad guy? what are the chances that if we use the nazi medical results, someone will start a land war with russia, to aid in their drug discovery programme?
How much extra work do we have to do to make deciding the right way about the incentives, or the re-use of results?

Why do I care?

Well, its clear that many open, public-minded people constrain themselves from doing potentially valuable research by setting barriers before working out the cost-benefit/incentive balance, whilst at the same time, a lot of industry just goes ahead and does it, and fixes things up after the event.
Is there a middle ground?

I'm suggesting that considering re-use and the safeguards one should put in place for that, might give a way to evaluate whether something is ethically acceptable or no.

It could also serve by offering "re-use cases" which might be easier to explain to people as part of the "informed consent" steps of any ethical experimental design.

As more pressure is put on us to do work with larger and larger data sets (whether the GCHQ surveillance data, or police crime/geo-loc data, or NHS care.data) we need to figure this out soonest.

Intellectual Property and Production - workshop in Law, 24.1.2015

at Wolfson College, 25 Jan 2015
Organised by Center for Intellectual Property and Information Law

Jennifer Davies introduced the workshop (6th in a series run by CIPIL ) and reminded attendees this was about production (and place!)

Session 1: Producing Output, led by Lionel Bently

Tanya Aplin: 'The Impact of Prosumers on Photojournalism' - this talk mapped out how the massive rise of amateur photography and the ubiquitous presence of cameras (especially in phones) had given rise to the de-skilling of the area of photojournalism, starting especially in disasters and emergencies (Katrina, Tsunami, Arab Spring etc), where classical photojournalists might not be on the scene for days in any case, but now supplanting every day coverage of events. The talk then went on to the new businesses that had arisen to "professionalise" the contributions by the public, and new agencies that would find takers for pictures and pay producers (albeit massively less than the famous photojournalists of yore).

Great talk - i think to be fair, the "professionals" always over-stated the skills you need to be a good journalist - many amateurs are massively better than many paparazzi - see my friend tim harris photos - he's a research leader at Oracle labs....this is also true of bloggers - for example my local kentish town blogger produces more timely and better job than the main traditional local newspaper (in my view:)

Alan Durant: ‘Kinds of Work in Copyright: a Semantic Perspective’

This was a foundational take on the triptych "Original Literary Works" unpicking the individual and combined meanings over history and showing how ambiguous the terms sepeaately and together were and still are. THis has important consequences for public and other stakeholders understanding of the relevant law!

Hye-Kyung Lee: ‘Media fans as new cultural intermediaries’

This was about fan contributed works with a new spin - the Manga translators - fandom who take japanese and korean comic art, and produce versions for other markets! Again (as with the first talk in this session) the emergent business models have first been squashed, but then embraced and extended by the original Manga production houses. Fans reactions vary (as you would expect) from approval to "uncool" - also, interestingly, cultural differences mean that Chinese reaction is quite different than other countries in this regards. I asked if anyone has tried personality profiling (c.f. psychometrics facebook app in cambridge) to see what sort of people enjoy these activities - is it mostly about fame, social capital, etc - inclusion - etc or do some really want to make it like their heroes in the original comics? or a mix (changing over time). Was reminded of two related stories. (Anecdotal) - 1/ The Dr Who fan contributed plots and characters were threatened with a lawsuit for copyright reasons, but then it emerged some of the writers of Dr Who had been reading the site, and may have inadvertently used material and fans suggested that they might countersue - so a quid pro quo was reached:) 2/ My cousin was teaching computer science in Brasil and translated US text books by reading then speaking aloud in portuguese, to a dictation/speech to text system (Dragon Dictate) - students bought the original book, so there's no loss to publishers or authors, but were given handouts of the text part in their own language.  Publishers became interested (low cost way to get foreign language editions of books....and discover markets).

This element of fan led A&R seems like an obvious way for producers of new work who live in the "long tail" of the popularity distribution to find their market, and for new fans to find content. Seems like a good thing - for a given area, it may not be a zero sum game  although over all content, I guess the population does have a finite disposable income. However, this new mode may lead to a wider breadth of availability, perhaps at the expense only of the few people at the very tip of the popularity (U2, Beyonce might lose a few percent of their gazillions) - maybe Thomas Pikety will approve!

Session 2: The Means of Production (I) led by: Ian Walden

Jon Crowcroft: ‘Software Tools and Die’
here's my talk which is mostly just about how we write software but also a plea not to do s/w patents (the clever bits are just math) and how copyright is ok, but there's lots of different uses (license models) which can all work (and co-exist)

Catherine Seville: “Controlling the Production of Literary Works: Hard
Then, Impossible Now?

This was a fascinating talk about the ways in which the technologies for production have always seen (mostly failed) attempts to control - from limited access to printing press, to control of rights to publish, through to DRM etc - really interesting...

Simon Eliot: ‘From 'Literary' to 'Commercial' in the Registry Books of
the Stationers' Company

Beautifully read and presented talk about this history and hilariously incompetent antics of rhe Stationers' company (registry for all things trademarkish) esp in the 19th century, which some fabulous graphic examples of Victorial graphic design for a wide range of useful, daft, and sentimental works. Sheer diversity of display is astonishing.

Session 3: The Means of Production (II) led by: Andrew Griffiths

Sean Bottomley: ‘Trade secrecy and production during the late
seventeenth and eighteenth centuries.’

Lovely talk about Worshipful socieiies and their control of their magic pixie dust techniques-  interestingly, the use of sanctions against people for revealing these tricks to "foreigners" (i.e. non guild members) was relatively rare in practice.  It makes you wonder why the Worshipful Company of Information Technologists was set up - What have they got to hide???

Dev Gangjee: ‘Why Wine is Not a Commodity: Geographic Indications,
Place and Process’

Great talk as with all the talks, really well presented and clear) on "terroir, and the way GIs are being used more and more, and yet have very little public recognition yet (outside of the original "Appelation Controle) - I asked the questions a) how "little britain" is all this and b) can you have a GI for "produced near here" for any definition of here (i.e. eco-didn't fly far food) - answer, yes...

Theresa Lopes: ‘Trademarks and Production of Consumer Goods from an
Historical Perspective’

Great econometric analysis of a large body of data from several places showing how IP protection trends track a) innovation b) the economic growth/shrinkage - a lot to take in - hopefully, we get some slides/paper post workshop!


Session 4: Geographical Delineation, led by Session leader: Graeme Dinwoodie

Sun Thathong: ‘A Marxian interpretation of the traditional knowledge
debate: the role of IP in the
coexistence of primitive and capitalist modes of production.’

Fun stuff -not quite as alarming as it sounds - basically, traditional knowledge is folk lore and crafts embedded in simple societies and held in common. It is being ram-raided by developed countries profit led companies (most obviously, plant medicines in rain forests being patented by big Pharma). This is about the moves to try and figure out a range of IP models that will appropriately recompense those "simple" societies (which, if course, are never so simple). Main bug is the hilariously naive Marxian description of primitive communism, which an anthropologist would have a field day with (literally and figuratively), but the model economically is good.

Here I had to leave to go to a memorial Mass for my brother in law, Declan McKeever
so I missed one of the organisers papers, which was a shame as it was

foundational to the main purpose of the workshop:

Jennifer Davis: ‘The End of Trade Marks as an Indicator of Place and

the Implications for Labour’

Graham Dutfield: ‘The compulsory working of patents and developing

countries.’

Henning Grosse Ruse-Khan: ‘From Local Manufacture Rules to Import

Barriers in Global Production Chains’

Science and Policy - Why should they pay attention to us?

Reading the latest CSAP (center for Science and Policy) report on "future Directions for Scientfic Advice in Whitehall" is quite frustrating if you are a geek.

Several behavioural weirdnesses define geeks, and these matter:

1. geeks tend to read about anything whatever their core training is, whether climate science, social media analytics, psychology, behavioural economics, and XKCD
2. notwithstanding Steve Hand's memorable pub rant[1], politics appears to be quite a lot simpler than a physics (Natural Sciences) or computer science or engineering degree, really.
3. they witness many people in political decision making roles who are actually less qualified even in terms of soft (social) sciences (lets not rehearse the PPE pub rant again just yet [2])

So what does this lead to?

1. incredulity when governments do not act on scientific advice (drugs, immigration, climate).
2. frustration when governments offer explanations as to why they cannot act on said advice.
3. strong inclination to walk away from bothering ever again to offer advice/evidence.

Far from this being something scientistcs should apoligise for, given the nature of the public's alienation with current western democratic policies (e.g. economics of austerity), it behoves politicians to rethnk how they react to public advice:

1. scientists have a methodology (atually lots, but lets stick with Popperian classical Object Knowledge for now).
2. Theories are falsifiable - the latest theory is "best of breed", that's all its merit is.
3. We dismiss theories when new ones come along that meet the criteria (better fit, simpler, more elegant -- pick any).
4. We change our minds. 

Indeed, there are honourable occasions (e.g. discovering that stomach ulcers were caused by bacteria, or the italians finding that their neutrinos did not go fast than the speed of light, or the cold fusion fail), when this is very public, but it happens every day on a finer granularity

I think the time has come for scientific method to be applied by politicians. Just as Jeanette Wing argued for computational thinking to be part of everyone's intellectual landscape, scientific method has already been embedded in everyone's subconscious for some time (e.g. since the Age of Enlightenment, aka Age of Reason)

Why not? As Oliver Cromwell said to one government (and remember what happened to them 

Think it possible you may be mistaken.

CLuDo

CLuDo

All bar one of the suspects was ruled out through a rigorous process of logic
and fermentation.  Suspect B was never identified, although her love of Citrus
flavoured chewing gum inclined Devonshire to discount her on the basis of the
lack of scent in any of the crime scenes. The fine detective had the measure
of Napier, and deemed his extremely asymmetric length arms to put him beyond
the reach of the law.  The recent Turkish immigrant who went by the moniker Moosh had an alibi (waylaid in a public loo by a nearly terminal bizarre
kebab accident). The Polis had no beef with Professor Baron, who was in any
case too absent minded to remember who could be next o nthe serial killer
list. Ms Bath was in the Ale House, where she tended to hang out with other
birds in hand. Dr Burleigh was too gentle a character, despite being a well
known Arms dealer.  The reverend Carlton was left handed, whilst the
Carpenters were both right handed, which saw them in the clear.
Thomas Champion was in london rowing, on the  Thames that day, and indeed,
had turned a corner since leaving the county seat for town, and given up his
unnatural addiction to Dobblers. (Dobblers is that well known breakfast dish
that looks like a classic grill, but in fact is made from dog & pheasant
and isn't as illegal as it is unpleasant).

The Earls of Beaconsfield and Derby were both out at a shoot, and so the
inspector was shot of them, even before five bells. Indeed, he thought to him
self with a wry ha-ha, pigs might fly and his golden hind come in, before
they'd commit any such gross moral turpentine. Henry was OK, since he'd made
peace with his Armenian girlfriend Ishca over the proper use of feathers in
millinery.

It was known that Milton was blind drunk, and waxing eloquent over some
jolly waterman about his time playing frag piano in old Orleans.
The chancellor was above suspicion, as were any of the occupant of the
Regent house. All five of the Irish brothers were known to react badly to the
men in blue, a red rag to a bull, but the inspector rose to the challenge, and
avoided a crowning.  Rupert Brooke was with Milton the whole day, according
to the sheep rustler known to everyone just as The Eagle, who had dropped in
after some wild spread betting.

Radegund's sainted aunt claimed he was down at Ally Pally, his Alma mater, and
the only anchor in his peripathetic life.
The inspector would give drink bakers dozen once the moon was over the yard
arm if that blackamoor's head was capable of such dark thoughts.
Devonshire asked his trusty sidekicks Argyle and Clarendon if they thought
there was anything in the rumour about the boats? Dr A saw on the
British Queen not, whilst det inspector C, smelling like a
brewery concurred. Devonshire looked at these two interchangeable chaps, like
peas in a pod - you couldn't tell if you swapped them, castle-to-castle like.
He hadn't graduated in the Jubilee year to waste time like this. He had other
buns in the shop to fry.

Back in  Cambridge feeling blue drinking a hog's head of big black cow and watching the
cricketers cut down another elm tree, failing to impress, as they fought
by George to keep the greyhounds and haymakers at bay, Inspector Devonshire
wondered if perhaps the evil genius artist, Geldart was behind it all.
Granted, he had the locomotive and was a man with a means, if not completely
in a pickerel. After all, he was known to have a predilection for chasing the
Green Dragon down King Street at quite a run. Many times, Dr Kingston had
tried to wean him off, but he would claim to be a master of the Marinade,
although everyone knew he was just driving them up the Maypole.

So summing up the first victim, Panton was found  bashed in by a
globe carved curiously with Fleur De Lys; the second, Portland,
rolled around a Fountain, pursued by a green manatee. The third,
Ms Parrot, crushed beneath a free press, like grapes, blood
spoiling all the nice mitred corners, pouring out like an old spring, only
attractive to the rats. The fourth, Salisbury trapped between the mill stone
and a rock, but curiously tied up with rope and twine. The fifth was drowned
regally when his ship ploughed into a dead end.

At six bells, listening to Bert Jansch playing
down by black Waterside on his iPod, the inspector suddenly remembers Arthur C Clark's story from Tales from the White Hart
(the one with all the violations of Sir Isaac Newton 3 laws of droog).
he lit a fine Unicorn cigar with a White Swan vesta matches, and decided to
see if the Wrestlers were still slugging it out over a bacon, lettuce and
tomato roll. As seven starts came out, the inspector took the last tram home
but fell asleep and landed up in the depot once again.

principles of communications, 2013/2014, end of week #7 to L22

This week, we covered shared media & ad hoc capacity, and started on traffic engineering.

A sharp question on proportional fairness in earlier material prompted me to notice that that isn't well contrasted with max-min fair sharing -- It turns out (as often with technical areas) Wikipedia has a nice explanation - see this article on
proportionally fair w.r.t weighted (max/min) fair queues

Next week will finish traffic engineering and wrap up with summary of course.

principles of communications, 2013/2014, end of week #6 to L19

This week have done scheduling, queue management and switching
and just about to start on shared media

One interesting point historically -the colossus computer at Bletcheley Park built for code breaking was not a von Neumann classical architecture computer but was a "switched programme" machine  - this made it incredibly fast (for a 1940s design) although incredibly inflexible -- and it took a very long time for people to catch up on a standard desktop (about 50 years) - amusingly, about as long as the Dr Who series has run on BBC TV:)

principles of communications, 2013/2014, end of week #5 to L16

This week, control theory and optimization...

Some minor inaccuracies in slides have been corrected in the online copies linked from the course materials page...[or will be as soon as I can get powerpoint with the right fonts:) - the key error is in the calculation of the steady state error of the proportional controller - for some reason, there's a subtraction of the two terms for U(s) where it should be +
(KUs + Rc) / (s(s+K)
I think {need to check this:) it kind of makes sense (if the completion rate increases, the admission rate should increase....)

then when we take the limit of s(U(s), as s->0, we'll get Us + Rc/K
so ess (error in steady state) is Us - (Us + Rc/K) which gives us -Rc/K
(i.e. the answer is right, but the system response wasn't...will check and correct soon...

again, to note, the chapter on control theory in Keshav's book is very clear if you want alternative source + some nice example problems.

principles of communications, 2013/2014, end of week #4 to L13

Error, Flow and Congestion Control done (99.9%)

further reading - maybe - on Network Coding (see Digital Fountains)
and on what's in Linux (CUBIC) and Windows (Compound) for congestion control, and what real traffic actually looks like - see CAIDA
http://www.caida.org/home/

next week: control theory...and optimzation:)

principles of communications, 2013/2014, end of week #3 to L10

This week we covered routing -

there's one egregious error on the slide explaining Dijkstra's algorithm in Link State where the sign on the comparison is the wrong way round (well spotted students!) - I leave it as an exercise for you to find, as it makes for careful reading:-)

In Sparse Mode, we use Rendezvous Points to coordinate a single RPF tree around a designated/configured router (maybe one for each of a different block or subset of multicast addresses) - there's no guarantee the RP is in a sensible place, although the switch from RP centric tree to source based tree after an traffic flows helps reduce latency -  automatic placement of an RP to be in the "centre" of the group would be a solution to the Steiner Tree (Min spanning tree) problem which is NP-Hard, although there are polynomial time approximation algorithms for it (but you probably wouldn't deploy them in routers, but in a network management system for e.g. a gamer or trader network, this might be sensible)

One other note - consistency, symmetry of routes, and so on - IP and IP routing make no guarantees about this at all! BGP (inter-AS routes) are often asymmetric...recent computer science work on building new protocols that provide global consistency during route update and computation does exist, but is still research, largely....although the techniques are promising!

Next week, errors, then flow and congestion control.

principles of communications, 2013/2014, end of week #2, to L7

To note for today- the slide on graphs, with Edge and Node list has a list of all edges, alongside o nthe right list of nodes  - the list of nodes isn't meant to line up with the list on the left - its just a list for node i=1-5, what other nodes, in the directed graph,  are adjacent (look at arrows on edges - note in 2 cases (1<->2 and 5<->4, they are bi-directional)....



fun references today:-
Ghost Maps

Collatz

Kirchoff

Erdos

Small Worlds...

DDOS visualised

Couple more corrigenda/errata
1. in the alpha/beta models of random graphs, there's k used for average degree of the net (e.g. pN in the alpha model), but also used for the toal number of edges (N*(N-1)/2) - so take care with k
2. there's an expression in the slides about max-flow in DAR (the "Sticky Random Routing" for the telephoen net) for using Erlang's call blocking probability for a given link, then work out what the toal capacity will be for 1 hop and 2-hop/tandem routes - this has n, which is number of calls you get through, then mentioned a technique called LP  to solve the maximisation problem given in terms of sum of calls that get through (or are blocked) over all direct and tandem routes- we are'nt covering that technique this year, but LP stands for Linear Programming, and is fairly straightforward if you want to look it up - it is commonly used in optimisation and shows up in Operations research/Logistics (freight etc) and so on all the time.

principles of communications, 2013/2014, end of week #1, to L4

We have now covered Systems, and Layers (lawyers)...

Next week, Graphs, and Routes!

principles of communications, 2013/2014, L1

Today this course starts: principles of communcations

This blog will be where I put errata, answers to questions, and just generally track progress of where we've got to for students and supervisors....

Today [11.10.13] got as far as 1/2 way through Systems lecture - Monday
intend to finish that and go about 1/2 way through Layers material:

slides

Publication Culture in Computing Research Design for impact: Rethinking academic institutions from the ground up

Why do we pretend that a publication is an event, rather than a part of an ongoing
process?
Computer Science is a Soft Subject. We create artificial systems/artefacts, and explore
their behaviours. We then report on this by talking about the behaviours at workshops
and conferences, and writing about the systems in papers for web pages, online
archives or even traditional print journals.
People assume that the artificial dichotomy between social events (workshops,
conferences) and archival repositories (journals and the like) is right. And some of the
debate about CS publication culture is oriented around trying to get people to use
these two modalities  more like other disciplines.
I think this is fundamentally wrong, and flies in the face of real scientific method.
Science does not deliver truth. It delivers things that work, and explanations that are
the best, current, simplest ones (c.f. Popper on Objective Knowledge, and of course
Occam’s Razor).
This means that a work is not the final word. It is just the current word. A goal of this
proposal is to reduce the “slice and dice” culture present today due to various perverse
incentives.
So the notion that an “archival paper” has been thoroughly checked and is infinitely
more “correct” than a “rapidly” reviewed conference submission is not tenable. There
is every chance that during the necessarily longer process to create an archival version
of a work, subsequent work has improved over the results. Hence much archived
material is actually less accurate because it is less timely.
The solution, for me, is to remove the notion of immutable publications, and admit
that we should update work continuously
This can apply to the entire process of socialising our work, hence a dialogue (or
multilogue) between authors, reviewers and readers, continually adds accuracy or
timeliness (or invalidates a work).  The same can apply to citations (which should, by
the way, have a “sign bit” to indicate whether the citation is building on fro ma work,
or citing it as the thing the new work invalidates).
Recognising this mutable publication model, would allow work to be presented at any
point along the “production line”, perhaps merely by “acclaim” - some work has
reached a point where it is mature enough and timely and interesting enough to merit
presentation at a social event (workshop or conference) - this could happen before or
after some notional point when it is recognized that an archival version is the current
best knowledge we have (a rare event).

Along side this continual process, I think one would have to abandon ideas of
anonymity in both authorship of work, and reviews/critiques (viz, the “dialogues”
mentioned above could only work in that open way). It goes without saying that code
and data associated with a systems’ behaviour should also be openly available as part
of this ongoing process (after all, since when did we declare code “bug free”
correctly? Why, therefore do we declare journal papers “correct”?).
Finally, this isn’t exclusive to Computer Science, but we built the tools that would
make the new approach viable, so we should use them first.
In fact we also have the next generation tools for this – we just need to combine Arxiv
with Github (versioning repositories)
1
.
Causes of paper count inflation.
CS is notable (in most branches at least) for submitted to conferences more than
journals. There are two pressures to do this
1. Urgency
2. Promotion
CS is a young disciple, and the young are noted for being impatient and impetuous -
our slogan might even be said to be “Publish Early and Publish Often”
2
.
Urgency
We live in a nanosecond world. More than other disciplines, partly because we built
it.
We supplied the tools and tool chains (the net, e-mail, the web, PDF, bibtex/latex,
databases, HotCRP/EDAS, etc)  that let us cooperate to develop ideas, systems,
results, and write papers faster, and deliver them for review, editing, and presentation
more quickly than any previous generation. Surely, other disciplines use the tools, but
we live and breath them.
As a result, there’s a feedback loop between publication of hot new work, This instant
gratification leads to an increase in the rate of submission.
Our profession has also a tendency (at least anecdotally) to attract a share of people
with OCD/Attention Deficit problems, who maybe (amateur psychologist’s hand
waving here) seek instant rather than deferred gratification.
                                             
1
 Github because we want distributed repositories to avoid re-concentrating power in
one place all over again.
2
 I could speculate here about whether these factors also contribute to the gender
imbalance in Computer Science as a profession and academic career (whether
directly, or simply as proxies for a root cause).


Promotion
Our academic research culture is funded largely by tax payers money (NSF, DARPA,
EU), and the tax payers seek metrics to see their money is well spent, and they seek
such feedback on an annual basis. Paper counts (and to a lesser extent, citation
counts) serve this. The same problem (inflation) has hit the industry research and
development world, where patents are a proxy for real work, and are rewarded.
The amount rather than significance of work is measured - hence, the aforesaid dice
and slice approach to work, producing minimal publishable units, and multiplying the
number of venues and publishable units year on year.
Because CS is young and vigorous, we have in the past been able to keep up with this
inflation. We are close to the limits though.
In the UK, we have a national Research Excellence Framework, for which researchers
in universities do not return all their work. Instead, every 5 years, up to 4 “outputs”
(e.g. papers) are returned. Secondly, and in addition, impact stories (pieces of work
10-20 years old, that have had a long term effect on the world, economically, socially,
or in terms of further developments in a discipline) are employed.
It will be interesting to see the outcome of this process, but for me, it is probably a
better basis for looking at some one person, or groups progress, so if we were to use
these sorts of indicators for tenure or similar, this would remove the aforesaid
perverse inventive to maximise the number of publications.
Acknowledgements
Thanks to Richard Clegg and Ioannis Avramopoulos for comments on this draft.

Principles of Communications 2012 up to L24 - week 8

Finished up with signalling, admission control, capacity planning today.

Comments on quantity of material and supervision q&a welcome 0- will work on this a lot for next year.

Was asked about reference (e.g. textbook) on WiFi  - not sure of good text (neither of Keshav's books cover this) but there's a nice tutorial online at Berkeley here

I'll see if I can find a better standard text on this, as it is quite interesting I feel!

Principles of Communications 2012 up to L21 - week 7

Have covered capacity of ad hoc wireless mesh, plus a bit on LP this week

As students pointed out, PR versus nPR is like comparing "reservation" and "staggered" forwarding in ad hoc mesh (i.e. pipelined forwarding is equiv to the first hop winner getting access to the whole path, whereas non pipelined case defers)

On SPT v. MST
http://www.me.utexas.edu/~jensen/exercises/mst_spt/mst_spt.html

xkcd has a (not so rare) educational cartoon on spectrum allocation which is useful:
http://xkcd.com/273/

LP - simplex solver - see
http://en.wikipedia.org/wiki/Simplex_algorithm

Principles of Communications - interim lesson

If someone wants to become a major hero, then fixing the Raspberry Pi linux USB/ethernet driver to remove "buffer boat" would be a very nice exercise - see here for references on what to do, and why - its an interesting lesson in
buffering, latency, and TCP/Queue Management interactions

http://www.teklibre.com/~d/bloat/Not_every_packet_is_sacred-Battling_Bufferbloat_on_wifi.pdf

Not every packet is sacred

Principles of Communications 2012 up to L18 - week 7

Scheduling

Randomness is your friend...see below too

Switching

should mention monsieur Clos!!
n.b. there may be an error in the slide on sorting/batcher switch - will check:)

Sharing

mention inventor of spread spectrum:
http://en.wikipedia.org/wiki/Hedy_Lamarr

A week full of s's...

Principles of Communications 2012 up to L15 - week 6

This week, Optimisation, and a start on Scheduling.

One question came up in optimisation - In the formulation of delay as
F/(C-F),  which I characterised as the load over the "headroom",
this is a dimensionless result - yes - its basically the average number of customers in an M/M/1 Queue (see Richard Gibbens slides from Computer Systems Modelling). However, in a work conserving router with a fixed speed output link, this translates into delay by multiplying by the
mean packet size/the output link line rate (which would have dimension time:)

Several people asked for more info about control theory - the chapter in Keshav's book (as per course web sight) is really quite clear (goes a bit past what I would ask, but knowledge is good, right?)...so recommend an hour reading that chapter - it also has exercises that are useful.
Keshav, S. (2011). Mathematical Foundations of Computer Networking. Addison-Wesley,
covers all but the graph theory bits of the maths I cover (and also covers some queueing and other performance things you might find useful as an alternative text, if you are attending Dr Gibbens' course too).

As per previous blog,
Keshav, S. (1997). An engineering approach to computer networking. Addison-Wesley
is also a useful text for the more protocol-oriented parts of this course.


Finally, if you are interested in the optimisation framework, then I recommend some of Frank Kelly's papers from the statslab - for example, this one briefly menions why we might take the sum of willing to pay times log of rates
w ln(x) 
as the  network view of the utility function..
.Fairness and stability of end-to-end congestion control

The two plots of functions of u_l (link utilisation of link l) are for
two different cost functions where the first one is exp(u_l)
and the second is n*(u_l^n) where n is a parameter (not the number of users - its just to generalise the function to a class of functions whose steepness/convexity can be varied by increasing n!

Principles of Communications 2012 up to L12 - week 5

Just made a hash of control theory....need to re-hash on monday to clarify- slides updated to show how G1 and G2 fit - see slide 23 on
control theory slides

Main point was to go through the decomposition of the control+gain+feedback
into separate boxes, to allow one to play with different controllers, and then re-compose to check the final transfer function for stability and for steady state error:- (slide update also fixes a couple of typos_

Hence, need to expand all the steps in the worked example with the video server and setpoint cpu load monitor

Will re-do on monday w/ additional steps for deriving the overall tranfer function in the s domain for the two different controllers of the CPU system whose basic (G0) behaviour is an integrator in time domain, so 1/s in Laplace transform/freq domain. this applies to the setpoint (Us) and the Mean Completion rate (Rc), so that when we look at these in the transform domain, we have the integral of them over time, which gives us a 1/s in the terms
for Us(s) -> Us/s and Rc(s) -> Rc/s


Looking at the slide where we first encounter G1 and G2, this is basically the design of a ne wsystem where G2 is what G0 was before (i.e. the video server modelled as an integrating service over time, but now with a new, regulated/controlled input), plus G2, which is the controller C, which has inputs which are the setpoint, and the demand, and outputs the new accepted/admitted flows which now go as inputs into our G2 (what was G0) who has an additional input, Rc(s) (or Rc/s).....

G0 = 1/s
Now add an (as yet unspecified) controller, C
and expand to the two stages, G1 and G2:

G1 = C.G0 / (1 + C.G0)
hence G1 = C/s (1+C/s) = C / (s + C)
G2 = G0 / (1 + C.G0)
hence G2 = 1/s / (1 + C/s) = 1 / (s + C)

Now our overall system is the composition of G1+G2, with
G1 handling the input decision, and G2 taking that plus the completion rate of work:
U(s) = G1.Uset/s + G2.Rc

so proportional controller just as C = K
and proportional-integral (PI) controller has C = K(1 + Ki/s)
where K and Ki are the constants to be chosen by designer:)


U(s) = C.Uset/s.(s+C)    + Rc / (s+C)       1.
which with C=K
 ( a proportional controller), gives
U(s) = K.Uset/s.(s+K) + Rc / (s+K)
Proportional controller:
stability: pole at s=-K, therefore ok.
error: lim of s.U(s)
= s . [K.Uset/s.(s+K) + Rc(s) / (s+K) ]
assume Rc(s) = Rc/s (i.e. Rc doesn't vary fast compared with feedback loop time)
= s.  [K.Uset/s.(s+K) + Rc /s.(s+K) ]
= [ K.Uset - R / (s+K) ]
which as s->0, goes to
Uset - Rc/K - so the error is Rc/K

For PI controller, put C = K(1 + Ki/s) in to 1 instead


G1=C G0 / (1 + C G0)
G2= G0 / (1 + C G0)

C = K(1+Ki/s)
G0 = 1/s

so G1 = K/s(1+Ki/s) / (1 +  K/s(1+Ki/s)) (* top and bottom by s^2)
 = (Ks + KKi) / (s^2 + Ks + KKi)

G2 = 1/s / (1 +  K/s(1+Ki/s)) (* top and bottom by s^2)
 = s / (s^2 + Ks + KKi)

response = Us G1 / s + Rc / s G2


....need to do this in tex:)

If people are interested in the stability of TCP's AIMD, then I have to say that its complex - to my knowledge, no-one has shown it for a network with FIFO "drop tail" queues, and heterogeneous RTTs - however, with an Active Queue Management system (like RED - see upcoming lectures on Scheduling and QUeue Management) there are some solutions - see
1. paganini's proof
and
2. INRIA work

Principles of Communications 2012 up to L10 - week 4

Made Errors:)

About to Start Flow Control. (Feedback welcome:)

Asked what books cover the non math component of PoC - answer is on the course web page - best reference is the other book by Keshav (An Engineering Approach to Computer Networking), which should be in most (college/lab) libraries.

Asked where to find proof of the bound on diameter of Erdos-Renyi graph - refer to this review/tutorial paper:
http://www.barabasilab.com/pubs/CCNR-ALB_Publications/200201-30_RevModernPhys-StatisticalMech/200201-30_RevModernPhys-StatisticalMech.pdf

Principles of Communications 2012 up to L7 - week 3

Couple of errata on graphs :

1.
s/walk/path/ in one slide (i.e. whether a vertex can appear more than once!)
2.
p<3 -="-" algorithm="algorithm" dar="dar" does="does" finding="finding" fraction="fraction" good="good" greedy="greedy" high="high" in="in" is="is" make="make" of="of" p="p" probability.="probability." property="property" succeed="succeed" sure="sure" that="that" to="to" triangles="triangles" with="with">
progress:
finished LS&DV Routing -
Coming Monday,  wil complete Multicast, Mobile
Wednesday, errors
Friday, Flow Control& Start on Control Theory

Principles of Communications 2012 up to L4 - week 2

Today, started Graph Theory (well, background at least)
Monday, will pick up on graph properties, random graphs, small world/clustering and searching. Then Next week, should cover most the Routing area.

For further edification and amusement,

Cambridge Networks Network

http://www.cnn.group.cam.ac.uk/

Erdos Bacon number
http://en.wikipedia.org/wiki/Erd%C5%91s%E2%80%93Bacon_number

Erdos zombies:
http://xkcd.com/599/

kirchoff geek traps
http://xkcd.com/356/

Principles of Communications 2012 L1 - week 1/2!

oops #1 - failed to spot 2nd box of lecture handouts - so they will be available on monday - apologies - mea culpa (not the admin fault)

oops #2 - 1kbps on the net is 1000 bps, because the k is from the sample rate (so a KHz refers to 1000 samples a second) but 1k bits (or bytes) in computer speak refers to 1024 bits (or bytes) coz its 2^10 memory locations or whatever

so being pedantic and wrong on slide 9 is a bit embarassing:)

Michaelmas Term 2012 - Start of Term & IET President's Inaugural Speech

Today (well this week) term kicks off and I'm teaching Principles of Communications to Part II, and Network Architecture to Part III and MPhil students.

Meanwhile, last night, I attended Andy Hopper's really excellent speech at the IET in London, where a host of stars turned out, including an MP who is an engineer (and female) and other luminaries to hear his very very good words on how to make things innovative - he used a lot of nice use cases, taken from his experience and others nearby (ARM, RealVNC, Xen and of course UbiSense) but he also made some very good high level points about UK industry (under investing in Research) and how to fix that, and about Universities (do LOTS of innnovation and let the market pick) and about government (stop the REF now - it served its purpose and is past its sell by date and actually probably damaging things now). All good stuff - I dn't want to stand up and ask a question but if I had, I'd have asked him about the new stuff we're doing with raspberry pi, Digital Life Foundation, OcamlLabs, and Computing at School, all of which have extreme "business model" approaches -
See IET TV of Andy Hopper's talk

might have been a good place to name check serial entrepreneurs like Ian Pratt and Keir Fraser (Bromium and Convergent.IO, post xen), too....next time:)

Two ideas
1. cheap opthalmoscope made out of toy microscope & android camera phone & some simple DSP code (on the phone
2. remote control for hearing aid using ultrasonic sound from the android phone (speaker can go to frequencies higher than you can hear, but hearing aid can hear them) so you can reset programmes for different environments
(both for me:)

3. Must talk to CCNx folks about Andrea lo Pumo's work on policy routes for content centric networking - ok, so we are ignoring "content value chain" but I think its more than just "cache flow" versus "packet flow" :)

Back to school.....

testbeds - its not what they are, its who they are

my experiences of testbeds (arpanet, satnet, dartnet, planetlab, onelab, umbrella, gini, etc etc) is that it isn't so much the technology and budget, but the cohort of people engaged that mark out a testbed for success or failure or damp squib.

but that's just my experience - what do other people say?

Principles of Communications - Week 7 - Nov 25

FInished COntention Networks, and Shared Media/Multihop Capacity, and just started Traffic Management.

Goal is to complete traffic management on Monday Nov 28th. And wrap there- LP is a step too far.

Will update on monday to describe which components are non-examinable. In general, see
the contents for the course,
here

Principles of Communications - Week 6 - Nov 18

Today, I wil finish the section on switching (covering routers as well as TDB and Space switch designs) - since we have "Silicon Valley comes to Cambridge" in the building today, its worth talking about the link between
Cisco, Sun Microsystems and Stanford University, then we can also mention the link with Granite and Google (Dave Cheriton) and Arista. Also, the early Sun 3 and CIscos were same M68000 multibus motherboard + ether*n + T1 serial line....alas, only sun ran BSD Unix, whereas Cisco wrote a low level executive called IOS (nowhere near as innovative as 3 years later when Apple wrote an Operating System for the iPhone and called it IOS...)....if routers had run BSD unix, the Internet might be a better place:-)

Next week, we;ll cover contention networks (shared media0 as well as capacity of multihop radio nets.

Principles of Communications - Week 5 - Nov 11

Finished Control Theory
and Optimzation Framework for IP/TCP networks

Next: Monday 14: Scheduling and possibly might get to Switching by next friday (18th).

Principles of Communications - Week 4 - Nov 4

Reached end of flow control

couple of very insightful questions about
1) reduce buffering in IP routers
ii) play with RTT by delaying acks in smart phones...to help redux negative impact of buffer bloat!

Monday - control theory
wed/fri optimization.

Principles of Communications - Week 3 - Oct 28

Got as far as channel model of errors after modulation/coding, and a basic intro to Shannon. [New copy of channel slides just posted that fixes a couple of errata pointed out by students - note, that specific material is non-examinable]

Starting Oct 31, Will finish errors, then move on to flow and congestion control.

Principles of Communications Week 2 2011

Today (21.10.2011), got as far as LS routing (having rather messed up explanation of DV).
Monday, will repair DV, and then cover
multicast and mobile.

I need to re-check the DV count to inf example isn't wrong....

Then wed/fri 26/28 cover errors...hopefully with less errors...

Principles of Communications Week 1 2011

I've just got up to the representation of graphs today (14.10.11) - see
lecture 4 including kirchoff - Monday, we'll do Erdos and Bacon.

SO we've covered Systems and Layers mainly, if you want to look at Supervision topics...

Escher Circuits and Perpetual Immotion

Followers of my blog will be aware of my discovery of circular wind patterns across Cambridge, that cyclists have suspected are always against them. For several years, I have taken advantage of this, and make my journeys out of phase with other cyclists
thus getting blown along in the right direction "for free".

Accidentally over the last couple of weeks I have discovered another phenomenon in Cambridge, which requires you to travel out of phase with the wind, but when there isn't any, and that is that there are certain routes which are down hill all the way there and back again. I refer to these routes as Escher Circuits after the great MC Escher's famous eternally descending waterfall (and the stairs in the library in the Name of the Rose of course, by the oft-copied inimitable Umberto Eco).

The existence of Escher Circuits has long been disputed since first suggested by the theoretical natural philosopher, H.King in his paper "Not enough string". The possible existence of Macro-circuits, measurable using crude mechanical devices was put forward in the seminal work by A. Hitchcock "Just enough Rope". But until now, these were mere hypotheses.

Of course, those of you who are students of natural philosophy will be aware that a naive analysis would dismiss such theories as contrary to the idea of conservation of energy, for surely, the cyclist pursing her cyclic route, would ever gain momentum.
However, my observations have shown that the real-world phenomenon is more subtle than the mind of man. While it is the case that the journey from A to B is downhill, as is the journey from B to A, nature, in her wisdom, has arranged the dimensions so that one arrives at A after a trip to B, at the same time that one started. Hence, time has flown backwards. And this is true no matter where you measure the progress of time - for any subset of the journey, for the return part, while you are on the 2D segment of the Escher circuit, time flows in the opposite direction, so you can take no advantage of the accumulated energy at all. A new branch of relativistic invariants must be supposed, not special, or general, but adversarial.

Thus Escher Circuits are rare, and exhibit adversarial relativistic time dilution.
Now, it is the case that one can make use of the properties, but only for a rather narrow application, and that is when one needs to use no energy to stand still in the face of a headwind. Of course, the hands of time and the wheels of the bike make the same amount of progress, which is to say, none at all. But you can get plenty of uninterrupted thinking done, which, after all, is the main reason we cycle everywhere in Cambridge anyhow, isn't it?

Week 7 - to Nov 26 - Got to Traffic Management.

Looks like we won't make it to the
Optimisation Theory and LP material this year -

I will wrap up on Monday 29th Nov
with last part of Traffic Management,
and an overview of what I've covered.

That will be the last lecture for Principles of Communications.

Students that are very keen can read the slide-ware on Optimisation and on LP - I am happy to answer questions on it too.

Students interested in the lower levels of physical/link layer may want to take the Digital SIgnal Processing course by Markus Kuhn next term (see here. Students interested in networking performance (and systems in general) may well want to go to Richard GIbbens' course on Computer Systems Modelling which covers a number of these topics in more theoretical depth. If really keen, please sign up for the new Part III, which will be running next year.

Note that next year, Information theory will be taught separately (again), which may make the amount of theory material in PrincComm slightly more tractable.

week 6 - to Nov 19 - Got to end of Switching

Next week, to cover
Shared Media
Capacity of Multihop net
Traffic Management

Then final week, will ust get to do optimisation and LP hopefully:)

n.b. to supervisors and students:- i've put a couple more links to some online information about
control theory, graph theory and some of the sources have worked problems...
see slides page for course
http://www.cl.cam.ac.uk/teaching/1011/PrincComm/ppt/

Week 5 - Nov 12 - end with Control Theory

Today, noticed that the wikipedia article on this is pretty good, but most especially nice is that it cites an 1868 Royal Society paper from the Royal Society by James Clerk Maxwell, which not only mentions Mr Watt's Steam Engine, but Mr J Thomson's experiments (noting that our building is between JJ Thomson Avenue and James Clerk Maxwell Road:)

Next week, we should cover
Scheduling
Switching
Shared Media Access


[The paper above also mentions a Mr Siemens!]

Week 4 Friday November 5th - Principles of Communications Progress

Today, we'll cover Queueing Theory. So that completes network layer stuff
(graphs, routing, errors, queueing)

Note in the printed (and old online pdf) queuing theory slides, there was a font error on some slides with \rho being rendered as ~n.
I've fixed it on the PDFs online (its ok in the ppt). apologies (again).

Next week we start on Flow Control, and hopefully get up to control theory.

Principles of Communications - End of Week 3

I have just about got to the end of routing
(having fixed, i think, some bugs in the distance vector worked example) -

next week
monday, wrap up multicast/mobile routing
then

error control
queueing
and maybe start flow control

Principles of Communications end of (full) week 2.

Today, I'll finish the graph theory lectures, covering social networks, small world nets, random graphs, alpha/beta and spreading/search.

So we'll have done:
# Introduction1
# Systems
# Layering
# Information Theory
On this topic, John Daugman's notes are great
# Channel Capacity
(not Modulation - this is on hold to end in case we have time)
# Graph Theory
# Social Networks+

On the last topic, this book on Small Worlds by Duncan Watts is a nice read. Another good book on the topic covers more about flows over such networks (information or diseases for example) is
Connected, by Nicholas A. Christakis and James H. Fowler

That means from monday (and most of next week, oct 25,27,29) we're doing Routing.
If things go to schedule, then subsequent week (nov 1,3,5) will be Error Contol, Queueing Theory and Flow Control.

Principles of Communications...end of first week (15.10.2010)

Should just have got up to first slide set on Information Theory (Entropy)
Monday 18th will start on Shannon - by 22.10.10 hope to get to Graph Theory.

Have just updated online slides (1up and 6up should all print ok now, fingers crossed:)

information theory - live example

powerpoint for lecture on information theory+colour printer -> slides without equations:(

powerpoint for lecture on information theory+mono printer -> slides with equations:)

ergo, colour printer driver is an erasure channel with memory and rather non random behaviour and information rate is massively reduced :-(

pushing this as an example for explaining shannon is a bit of a stretch...in the sense that the "physical channel" is the printer and the colour printer should have more capacity in some sense, although I suppose the point is that the "noise" process" is an erasure channel that removes (say) bytes that code yellow but not bits that code white/black...

In schloss dagstuhl again - 99/100 (1 small nit)

I really like dagstuhl - the organisation is wonderful - lots of lovely quietly smart tricks (for example, the librarian puts out copies of all the books by attendees for display and they are invited to sign them!) - the bar (wine) is fantastic (makes up for the so-so food - although the cheese platter is superb !) - the visitors book and other information for wider awareness of what is going on and who is there is excellent - the area is great - even the "interesting" challenge of getting here from the "nearest" airport or railway station is a fun problem

only 1 minor complaint - the website (in almost all ways very very good) has a facility for attendees to upload materials for their seminar/talk/wiki/discussion etc - but the site as a design flaw which makes it very counter intuitive to getting the first thing uploaded (when you navigate to My Documents, you MUST have a title and click on the very small "save all" button to avoid weird apache error messages:)

otherwise 999/100 for the best seminar/retreat/workshop site in Europe by miles

fine snooker room ,very nice music room, great cellar, good walks/hikes, pleasant and efficient (and mostly invisible!) staff....

2 weeks of workshops taking its toll!

I was at IMDEA for this workshop on energy in networks, then at this one at Microsoft on Network Economics, then another one at UKERNA on SuperJANET 6 research requirements planning at the very nice Royal Society Chichelery Hall Kavli center then there's this one at the Isaac Newton Institute on networkmeasurements, and one on new internet architectures and incentives rn by the Eiffel thinktank, then one at Telefonica I+D on a vision for 2020 networking, then this one run by the Trilogy Project on resource pooling

Boy, now I need a vacation:)

here we go again with government interference

without waiting for Lord Brown of Madingley's report, the new government make proncouncements about UK Universities - the BBC is reporting David Willett's saying he needs 700M savings and the coalition's evidence-free arguments that
a) we (the uk universities) are in a financial mess
and
b) that we have to find ways to be cheaper and more flexible.

Well we don't actually. We are quite cheap (which is because we have to serve the UK government) compared to a lot of EU and US universities AND we are evidentially quite flexible and quite good (compred to most EU universities) - the government should check the figures on health and education in the UK and compare them with similar quality systems in the world. We attract a lot of overseas students of very high quality preceisely because we strike both a bargain, and a high standard. We don';t have to serve the UK - we could just turn into private institutions for rich kids from here and overeas. Or we could be rubbish. But we are not inflexible or expensive.

Frankly, I'd held out a bit of hope that this new government would actually base its policy on facts, but it looks like I was hopelessly naive and sentimental.

What a shame

schroeding & turing

Schroeding's Cat is the typical name for a thought experiment about the oddness of the Quantum model of the Universe. It is an artificially constructed scenario where the normally microscopic quantum level effect selects a macroscopic change in the state of the Universe - i.e. the decay of 1 single atom triggers the life or death of a large creature (a cat is presumably on the order of 10^24 molecules) - Life or Death states of the cat are
quite visibly distinguishable to a human observer. Of course a simpler trick would be to have a critical mass of fissile material, less 1 atom, and then just add that 1 atom and then see if it decays or not and does so triggering the chain reaction or not....that would also be quite visible and not very distorted by emotive weird observable things (when is the cat dead? what is life? :-)

Turing's test of intelligence decides the matter based on a human observer's ability to distinguish the responses of a black box to a set of questions (sent over some teletype line to avoid any complex non verbal comms). It can be generalized.

Some confusion arises when thinking about Schroedinger's Cat ("paradox") and observers.
Note that the state of the cat is actually created by an observer (the geiger counter described in the wiki article above is the thing that detects the decaying atom, and triggers the hammer to break the flask of cyanide that kils the cat (probably). The uncertainty is purely about when the decay happens. An observer of the dead cat is a secondary thing - we can assume fairly certainly the cyanide kills the cat (all nine lives) so in fact all we need is a red light on the box that says the geiger counter detected the atom decaying. The confusion arises because two words are misused from natural language:

observer

description

The observer does not have to be intelligent.
The description is not obvservable.

Hence we can't use this to talk about consciousness, and there is no paradox (descriptions of intermediate unobservable states are figments of maths - only if actually observable are they interesting (e.g. remote entaglement).

The point of mentioning the turing test is that Schroedinger also conflates two things:

simple observation (detect decay from emitted particle

complex observation (is the cat alive).

Is detecting life, perhaps a nice turing test?

remarkable engineers

this is a fun read if you like these sort of mini-bio things :-
Remarkable Engineers follows similar excellent books on maths&science. But it is an interesting book for its lack of people that I would regard as engineers from Ancient Greece, Arabia, and China - however, within its well-defined model of what an engineer is, it is interesting (seems like you not only have to build something, you have to patent and sell it, and preferably be Scottish:)

1 mole of stars. Avagadro, state and human perception of scale

so reading
The Eerie Silence by Paul Davies,
he points out that there are estimated to be around 10^23 starts in the universe around now....this is an interesting number (like all numbers) which is around Avagadro's number which is the number of atoms in 12 grams of carbon 12
or roughly 24 liters of air at STP.

one amusing number to play with is the number of molecules in a glass of water, which is MUCH more than the number of glasses of water in all the oceans of the Earth.

So the number of stars is in between.

maybe, they are the same thing.

Conferences, Journals & unrealistic levels of Academic labour taken for granted

I was talking to some PhD students here recently about the relative merits of submitting their work to Conferences or Journals - a typical chapter of a thesis makes a nice paper

before you write the dissertation, it is nice to have it published in paper form, but to get the process done and dusted before the PhD examination comes along, you almost certainly have to send it to a conference rather than a journal because of the turnaround times - its 6 months from submission to presentation for typical top-league conferences, and more like 2 years for the equivalent conferences in Computer Science.

Reason for publications: confidence boosting (for student AND for examiners:)
and feedback is useful to improve work even if it doesn't get in...

Plan B (if things work out) is to submit a 9-12 page version to a top conference, get your PhD and submit the 12-20 page version to a journal after....reason for submission: archival version (maybe) and certain academic communities' career paths still value journal higher than conference (mostly, though, citation is lower...)

Some points to consider though
1. academics or industry research lab workers do all the work for conference and journals (run submission sites, do reviewing, do editorial work, do selection, do a large part of conference organisation etc etc)
2. a typical top class journal or conference paper represents about 1MY effort - typically if you include all the effort by a PhD student for a year, plus their advisor and any co-authors - in systems, it can be as much as 2-3 people's work
3. a typical conference reviewer might look at 20 papers in 30 days - so if they are really really insanely fast at reading, and on top of all the latest material (and not handing it out to their poorly as yet calibrated own students to review, or at least only doin this to help their students get calibrated or give them up-to-date feedback, but still doing the review themselves) if you are lucky, your paper gets 4 hours work on it at a conference
4. a journal submission might get a rainy Sunday first submission, and then several hours more over revisions, so you get a lot more detailed care and feeding in the process.
5. In my experience, examining a PhD takes 3 days work - if you consider the naive equation, that 1 dissertation is == 3 good papers, then this is about twice the level of attention a paper gets (at best)
6. Finally, a lot of researchers use long journeys to read a lot of papers (transoceanic flights are good) - this appears to have stalled for the last few days around this part of the world - I wonder what impact this will have on conference review quality?
On the other hand, maybe the mortorium on flying will reduce the number of weak or nearly pointless workshops ...who knows...

two mixed reality dystopias.

1. Gravity 3.0 didn't turn out to be such a good idea, did it Professor Sheckley - I mean superficially, the notion of an inverse cube law force so that people on smaller planets, but nearby, get a strong attraction and don't drift off into space, looks OK - but then look at the terminal velocity - not good for the robot ships landing delivering out food....gasp gasp
Of course, it wasn't such a terrible mistake when you compare with Gravity 2.0, the result of the Campaign for Real Gravity - attaching everyone to the surface of an apparently zero mass planet, by colored elastic looked cool - but we didn't reckon with people customizing their g-strings, to have different h-indexes - Hookes' law is amusing, but then when they started playing paddle-ball with the alien, and we had to tell them to turn down the space-trance remixes of Phil Collins and eventually called in the Cruel and Unusual Ludic Police, it took a turn for the worse.

2. The entropic viral pandemic of 2012 has run its race, and now we cannot rely on the value of bits to be discrete any longer. Unfortunately, it was detected too late to do anything about backups, and so we cannot state the meaning of any program or data with any certainty any more. Such diseases of meaning were unanticipated in the early days of the semantic web, so that elementary precautions, like repeated recall and re-enforcement of what we were saving
were not taken. Now we have to rely on humans to memorize entire sections of the Internet, including music, live performance of movies, and physical versions of VR games.

The cause of the emergent entropic virus will never truly be known (just like every other piece of what use to pass for online human knowledge) but it is suspected that it was the mean temperature of the time series of arrivals of Youtube videos, exceed Centigrade 451, the point at which binary systems move to a higher, ternary state.

Cooling the system down will do no good now. It would be like telling Schroedinger to open the box and finding one and a half cats, one half dead and the other half alive.

This has been a Horizon special

progress bars and stress

so i just have had the 3rd experience of the week when I want to carry out a routine task on some Win* box and it decides it needs to run a massive update and
to make matters worse, this locks out other activities
but the straw that broke this camel's back is the
progress bar

the example of the service pack update for vista will suffice to show what a load of stress this causes - the bar is in %age complete - but there are 3 seperate phases and you are given no indication of their relative longevities nor does the bar actually
move linearly w.r.t time in any fashion (not even relativistic) so if you go away for what ought to be enough time (e.g. cup of tea or watch a movie) you come back and inevitably either something finishes 10 times earlier, so you now have to go to the next step, or else it still has a full 3-day test match (cricket) to go ....

this is like being in a travel warzone when the train/plane is late and the announcements are intermittent and noisy (or in a language you don't speak)...

The Law in 1995

I found this proposal I wrote in 1995 for a programming language for a sort of active web called The LAW - A Language for Agents in the Web

It kind of presages stuff like map/reduce and declarative networks, n'est ce pas?

I am a gadget, you are a widget, they fidget, bridget - Its the Long Gadget, Gromit

Jaron Lanier's fine new book,
You are not a Gadget has a lot of things to say that I agree with.

Turning it up to 11, there's a nice chapter suggesting some new models, which might very much fit the Horizon Digital Economy project - there are three notions, which I have re-branded as follows:-

1. songlets - his idea is more Ubicomp than mine - basically, create a
"songle" - this is a physical key to a right to play a song - could be an Rfid, or a Barcode on a can of coke - it creates an "artificial scarcity" for songs so that instead of people trading them for free on filesharing nets, people trade them for
micropayments (possibly bundled with other things)...its a nice idea
2. giglets - telegigging is the idea of inviting a live band to your party over the net - giglets would be my old version, which is that all live gigs should be webcast and recorded (a la bbc iplayer, NOT a la youtube) with low cost DRM - possibly using a model where its pay per view up to some deadline, and then switches to free if not moving out of the long tail, but staying pay per view for longer if it gets popular and moves up the zipfian...
3. marklets - these are instruments (in the trader sense) which can be formally checked for sanity against a policy + safety rule-base - this is a good idea and resembles some of the things George Soros- wrote in his fine work on the Crash of 2008 and What it Means - he put in some meta-rules (some are basically stability conditions in feedback systems with many actuators - some are what a Programming Languages person would recognize as reflection - all nicely and intuitively explained).


Things I disagree with:-

1. music and midi

- kids share music - they don't share midi, but they do share
recordings, Youtube videos, tab and formal classical notation
so that they can learn stuff - kids I see doing this
do play amazingly better than kids when I was their age - they
don't know they aren't supposed to be that good yet, so they just do it

The real problem is that any abstract representation of music (i.e. notation) degrades it - classical notation, tab etc

using printed symbols for notes is (to reuse an Elvis Costello phrase) like
dancing about architecture

kids i know (in the 12-18 current cohort) share the recordings of the results
not midi - maybe its coz uk is guitar-live oriented, or maybe i am ignorant of dance culture (well, 20 years out of date on it)

midi is so 70s/80s/ synth/drum machine - agreed - but that's only a trap
someone who is synth/keyboard/tech obsessed would fall into


2. code and inflexibility -
i) I was amazed not to see Larry Lessig cited...
ii) I don't see files as lock-in - that is like saying
bits are lock-in - files are just ways to frame a set of data - you can have sets of sets , etc - just like bits. there's a load of theory why this is general....

this is a granularity/sampling error, which i think is a mistake as a metaphor for a cognitive framing error - the notion of Unix files as a problem is much less weak, but still not as good an example as midi, even if I believe midi is a red herring...

3. on music
Doh, how dumb, the internet
Ray, a blade of golden sun
me, an apple trademark bet
far, site, neat place to keep one
sew, a cpu scheduling threads
la, a loony tune land of cgi fun
t, a pipe with 3 ends
and that brings us recursively back, doh

good, bad or midi?

4. singularity (c.f. Singularity Sky, Accelerando etc) and
techno-totalitarianism (none so bad as Extropian!)

The idea goes back a long way - Indian and pretty much
every other early myths have avatars and nirvana,
although, perhaps less blue and with less grunge...
to sublime, to get to cloud nine, to reach Twoness (oops, sorry, Woody Allen:)
or oneness - read Lord of Light....

5. upload and the Turing test - yes, its reductionist - Turing
wanted something akin to what Shannon had for information theory of a channel - so
he constrained the situation - of course it then lacks context, state, emotion, society etc - yes its sad.


6. Some more thoughts on this as I read about his critique of the impact of the "free to air, paid by advertising" business model - he uses examples from
journalism and music and berates the long tail argument


However, the death of journalism long pre-dates the Internet, although it is a technocentric death - the centralisation of press by barons such as Murdoch (and before, Maxwell - c.f. Wapping etc etc) destroyed most of the community of local newspapers and full time journalists who gave detail and colour - this is well documented - the book "Flat Earth News" shows the fraction of news that is actually written anew and analytically by journalists and the vast majority that is simply a verbatim reproduction of press releases. The complaints by media that Google gets revenue from avertisers simply for "indexing" this stuff is rediculous, since the majority of the material isn't even copyrightable by the so-called "publishers" in the first place - it is effectively plagiarised anyhow. (often literally without citing the original "source").

secondly, the music business has concentrated for 20 years on back catalog - this is why it has seen a demise faster than film - the problem is that this is the exact OPPOSITE of the long tail - the claim that A&R men invest a significant fraction of profit in finding new talent is pathetic - it is about as pathetic as most drugs company claims that they spend a large fraction of their profits on new drug development - they don't - they spend more on advertising. One obvious consequence of this is that kids are overwealmed with the quantitiy of high quality material they can see - if you are setting out to learn to play (remember first 3 issues of the first and greatest punk fanzine, Sniffin Glue covers: "here's a chord". "here's another chord". "here's a third chord - now go form a band") - this is just not going to occur in the retro world....on the other and, myspace is full of some cool crazy dudes - i would comapre the clunky wonderful amateurism of, say, sulek, with the madness that is harmelodic metal tapping, sweeping and shredding of Buckethead...

The Internet is a corrective to these poor trends - that doesn't mean it is a perfect replacement, but it sure is better than the alternative.

I love the idea that language might have started as swearing about smells - that is really cool.


otherwise 11/9:)

research is wasteful

reading about the government's reaction to the recession (well at least in its own (our) coffers), we can see that it is highly likely they will cut funding to research over the next year(s).

there are two ways to do this - both wrong - one is to spread the reduced funding thinner, the other is to concentrate funding in fewer (elite) places (actualyl 50% of the money goes to the top 5 institutes anyhow).

But what they should do is spend more. Research is wasteful, very very much like evolution - but necessity is the mother of invention and (just like in evolutonary systems) the impact of a big recession might make people more creative - more creativity does lead to more potential outputs, but it also leads to LESS predictability - we don't know what will fit the future landscape (i.e. what is "selected" as most fit, is not something you can do a priori, but a posteriori, only).

No doubt they will get this wrong, which is sad, as doubling the UK's governemt research funding would still leave it at under 2% of the bank bailout last year, and would be likely to yield more profit by about an order of magnitude.

Unfortunately, we are not ruled by people that have a clue.

clown computing

its so much more fun, and easier to market than the cloud

send in the clowns

i've looked at clowns from both sides now

clown atlas, using the Grimaldi internet coordination system based on
empirical data from Big Foot

CRU/Dr Phil Jones are a bigger pain than maybe they realize

I' reasonably well trained in natural sciences and computing. So I usually trust scientists to do their job as well as they can. In medicine, there's a whole bunch of due process before they loose a new procedure or drug on the world, and even if the big Pharma might be a bit dodgy (and most are fine), the process stops rediculous things happening too often (unless politicians get in the loop).

WHen I or a close family member or friend get ill, I do read the literature (the tech. literature, not just online freebie dodgy internet health sites) to figure out what is what - when i spent a week in hospital with metal in my leg, a couple of years after my father died in a hospital from an MRSA infection, I read all the latest papers on that - I didn't find it too hard to catch up and it mattered to me - but I assume, most of the time , that the latest practice is ok....so I don't have to read the 1 paper published per scientist per month in the world (it.d make me, on average, the 2nd reader only on average).

This is true in most areas where there's a large scale deployment of some piece of engineering (planes, trains, automobiles) too.

So even though I could get my head around a lot of the work, I assume I don't have to.

So the consensus on climate change has been fairly overwealmingly behind the anthropgenic global warming explanation, leading to many government initiatives and international schemes to try to get people, society, industry, nations etc to reduce carbon emmissions. Seemed ok to me - the bit I like was the sane stuff which looks at being, simply, more sustainable, anyhow, as that seems like a good way to conduct oneself - not having had a car for 45 years and having got 3 kids to/thru school on bikes- having an efficient house, etc - I feel ok - then I feel like I should carbon offset my flights (I used to do more, but I still, as an academic, go all over the place for project meetings, conferences, PC meetings etc etc etc)....

so now, because of the CRU fiasco (not their fault the email was leaked, but it is there fault the data isn't avaialble and in good clean shape) I now have to read through the mountains of literature on this topic to try to figure out what is sane and what isn't.

You know what is scary? In the 21st century, there is absolutely no way to tell what is authoritative .

I know what/who is serious in my own area. In biomedical area, I can usually guess to some extent. But in this area, it is completely impossible to determine

so not only do I have to read the literature, it looks like I might need to go out, and audit where the data comes from, then get ahold of some of the data, and then do my own analysis.

that is rediculous, but without doing it, I really don't see a way to have any confidence in the statements by AGW proponents, or by climate skeptics.
I am sure most of them are genunine. But none of them has a clue how to instill a sense of public understanding of why we should listen

this does not make me a climate skeptic (not at all) - it just means that for the first time, I'm looking at an entire body of so-called knowledge, and while I can grok the concepts, until I can grep the raw data files, I am basically saying that this is a complete crock!

mandy cuts his knows to spite his feys

1.
Mandelson said: "We know that universities have a vital contribution to our economic growth, so we are not going to undermine them. We are asking for savings of less than 5% and we expect universities to make these in a way that minimises the impact on teaching and students. I am confident they will."

2.
Mandelson said: "I am an arts graduate myself. We don't dictate to universities which courses they put on. They tailor courses to meet demand. We want universities to play to their strengths, but we also want to keep this country civilised."

since the universities contribute net profit to the UK, cutting them decreases the UKs GDP. so cutting them ANY percent will have an impact both on students and on everyone else.

of course, being an Arts graduate, Mandelson is probably not numerate to understand this.

doh

4 colour map problem solved....

...automatically -
turns out just to be a
bunch of
corner cases

:-)

now on to fermat's.................................................................................

Big Scary Software - python runtime, filesystems, etc

Mort (now at Nottingham) pointed us at some educational videos in the last couple of days

1.The Giant Interpreter Loop that python runtime uses is not really something that admits of concurrency, but also has a lot of weirdnes about it...
(sorry - as per comment - GIL = Global Interpreter Lock, not Giant Incense Lamp)

2. Making sure things really are written to stable storage or disk, as we used to call it, is really not quite as simple (or obvious) as you might think

3. not even on windows

I guess these should be like public health warnings, a bit like the German and French governments recent IE warnings