Proceed systematically by reading such that you find answers to the following questions write a review answering each of the following questions.
Please make sure to have appropriate subheadings for each question.
– Origin of the paper
• Where is the paper coming from?
• Research lab
• University
• Company?
• Is the lab / authors famous? If yes, for what?
– Network setting
Is there some kind of network in the paper?
• What type of network the paper considers? Wired, Wireless, Cellular, Sensor, Vehicular, Internet of Things, Edge/Fog, Hybrid System, etc.
If the paper considers a specific application area such as healthcare, industrial settings etc. identify it here as well.
– What is being improved
Most papers provide some kind of improvement compared to the state of the art.
• There is a very small number of papers which introduce completely new application
areas. But most new protocols etc. are introduced in order to improve some parameters
within an existing application
•What is the network (or other) parameter that is being improved?
Bandwidth, Latency, Reliability, Energy consumption, Cost, Privacy, User satisfaction, etc.
– What is the improvement technique
What is the technique used by the work to improve the chosen parameter(s). These can often be AI/ML techniques.
• new network protocol (MAC, routing)
• Search
• Linear / non-linear optimization
• Deep neural network
• Game theory
• Genetic algorithm
• Other (what?)
If the improvement technique is not obvious, should do some superficial research on it, and add some information to it
– When is the improvement applied?
• Before the system is run – For instance, static paths, planning techniques
• During runtime E.g. routing or MAC protocols
• After execution – For instance, for analyzing the logs.
• A related question: where is the improvement code running – On the routers, hosts, IoT devices, user desktop, cloud?
– How was the paper evaluated?
In order to believe the claims of the paper, we need to see how it was evaluated. This often involves comparison with other techniques. There are several ways to evaluate the claims:
• With theoretical proofs
• In handcrafted simulation
• In a publicly available simulation framework
• Testbed
• Real world deployment
Related question: what does the paper compares against?
• Strong claim: current best techniques
• Weak claim: technique is better than random
– Afterlife of the paper
Especially if the paper is older, it is useful to investigate its impact.
For instance, the Bill Gates paper is a mathematical curiosity.
• Gates, William H., and Christos H. Papadimitriou. “Bounds for sorting by prefix reversal.” Discrete
mathematics 27, no. 1 (1979): 47-57., citations: 369
• Another instance The Larry Page / Sergey Brin paper describes the algorithm which built Google.
How do we evaluate impact?
• Number of citations (look it up on scholar.google.com)
• be aware of the publication year, new papers might not have a lot of citations
• Was it deployed? Where? How wide?
• Is it the basis of a follow-up work?
– Personal view on the contributions
What is your opinion about the paper? Do you find it impressive?
• Brilliant idea
• Impressive amount of experimental work
• Deep theoretical work
• Convincing practical application
• Impressive improvement
Or maybe you are not impressed?
• Results slightly better than random?
• Comparison only with variations of the proposed algorithm?
• “Proof by intimidation” – excessive amount of math, mostly definitions,
obscuring the real problem.
• Contrived, unrealistic problem setting?