TRANSCRIPT: Lecture 1 of Hard-Boiled Synthesis by Marc J. Lajeunesse 
UPLOADED to YOUTUBE: Dec. 2, 2020
https://youtu.be/rM4MQA5hU6c


00:04
hi
00:05
welcome to hard-boiled synthesis my name
00:08
is marc lajeunesse
00:10
and this course is all about
00:13
trying to synthesize a research topic
00:16
using a bunch of tools
00:20
called systematic reviews and
00:21
meta-analysis and these tools form the
00:24
backbone
00:25
of a lot of empirically-based research
00:29
where we try to reach a consensus on
00:32
the outcomes of many studies
00:36
my goal for the class is to cut through
00:39
the theory
00:40
go straight to the practical and
00:43
actually show
00:44
you guys all the
00:47
funny decisions we need to make when we
00:49
try to complete
00:50
a research synthesis project and so this
00:54
course isn't about
00:55
learning about say "publication bias"
00:59
very important, but not about those types
01:02
of
01:02
concepts but to actually show you guys a
01:06
worked example of a
01:09
project and and so let me step back a
01:13
bit
01:14
and tell you guys a story just as a
01:18
primer for what to expect for the class
01:20
so years ago
01:24
i completed a project with a bunch of
01:27
undergraduates in my medical entomology
01:29
class
01:30
where we synthesized um
01:33
i forget how many studies not really
01:35
that many studies 16 17 studies
01:38
that looked at whether or not infected
01:40
mosquitoes responded differently to
01:42
repellents
01:45
and um i love the project because we
01:47
were like 30
01:49
people um mostly undergrads
01:54
working together to collate the studies
01:57
code them
01:57
and then analyze them and and we managed
02:01
to publish this study
02:04
and what happened when the paper started
02:06
to come out
02:08
um i was very excited about it and i
02:10
tweeted
02:11
about the project um but what i did was
02:15
also kind of like uh
02:16
sniffed around twitter hoping that other
02:20
people would uh
02:22
tweet about the project
02:25
unfortunately nobody really did that
02:28
that's okay
02:28
i'm totally fine with that but
02:31
here's the thing
02:32
something came up over and over again
02:34
which i thought was totally hilarious
02:36
when i was doing my twitter searches
02:38
right because i would search for like
02:39
mosquitoes repellents these were key
02:41
words
02:42
uh for the our project and this one
02:45
thing kept coming up over and over again
02:49
which was this catnip
02:53
is ten times more effective than deet
02:57
as a repellant to mosquitoes
03:02
that's quite the remarkable claim i mean
03:05
ten times more effective
03:07
than indeed if that was true wouldn't we
03:10
be covering our bodies with cats right
03:12
now
03:13
i mean i mean uh i mean uh
03:16
cat dip covering our bodies with catnip
03:21
i mean i live in florida and we have
03:23
mosquitoes year round
03:26
if catnip was really that much more
03:29
amazing than deet which is by far the
03:31
gold standard of repellency
03:34
um everything would be associated with
03:37
catnip
03:39
you know you'd go to the store you'd buy
03:41
catnip
03:42
repellents you'd have catnip growing
03:46
on your lawn i mean so this is kind of
03:49
an unusual claim and so this is what the
03:52
focus of the project is going to be
03:54
i'm going to synthesize all the projects
03:58
that use
03:59
catnip as a repellent
04:02
in order to um to reach
04:06
the conclusion or to evaluate
04:09
whether or not it's ten times more
04:11
effective than deet and the neat part
04:13
about it is there's an actual magnitude
04:16
in the prediction that is 10 times more
04:18
effective than d
04:20
and so i could actually test that and
04:22
how do you test that
04:24
research synthesis the whole point of
04:26
the
04:27
course and so
04:31
yes i don't know how many
04:34
lectures this is going to take
04:38
but i'm going to go through the entire a
04:40
through z
04:41
of the research synthesis practices
04:45
from say finding studies extracting data
04:50
analyzing writing up the manuscript and
04:53
submitting for publication i'm going to
04:55
do i'm going to do the whole thing
04:58
for you guys again just so that you guys
05:01
could see
05:02
all the silly silly decisions that are
05:04
made in order to achieve that final
05:06
product
05:07
and that's really what you don't get
05:09
with a typical meta-analysis course
05:12
typical meta-analysis course you go
05:14
through a lot of concepts
05:15
maybe you read a bunch of published
05:17
meta-analyses
05:18
but you don't really get to see or
05:21
experience
05:23
all the funny stuff that's involved in
05:26
actually generating the final product
05:28
the final meta-analysis
05:33
and so um let's go quickly go on twitter
05:37
i'm going to keep a fast pace with this
05:39
because i don't want to spend all my
05:40
time on this project
05:42
and i think it's you know it's a very
05:43
tractable just an aside here i think
05:45
it's a very tractable project because
05:48
there can't be really be that many
05:49
studies
05:50
looking at catnip right
05:54
okay so i already did a a twitter search
05:57
here
05:58
to show you guys um
06:02
how frequent this proclamation of 10
06:04
times more effective
06:06
than deet okay a 2010 study researchers
06:10
found that catnip was 10 times more
06:12
effective indeed
06:13
okay funny proclamation there's no study
06:17
linked with this
06:21
um there we go
06:26
researchers discovered that volatile
06:28
oils from catnip ten times more
06:30
effective at repelling mosquitoes than
06:31
deet
06:32
okay that's fine let's keep going
06:35
now the dates for for these
06:37
proclamations go way back
06:39
i mean sure we got stuff from march
06:41
right did you know
06:43
catnip is actually 10 times more
06:44
effective as a mosquito
06:47
repellent actually did you know
06:51
um and it goes on and on
06:55
you know we got
07:00
2018 nothing's loading really good right
07:03
now
07:04
there we go we got a mosquito
07:05
researchers discover new pesticides okay
07:07
that's
07:08
we got a weird uh link here catnip ten
07:12
times more effective and deep but the
07:13
link is actually about west nile
07:16
rip please ripley's believe it or not
07:20
tweeting out to the world catnip is ten
07:23
times more effective at repelling
07:24
mosquitoes than deet
07:27
ripley's let's just quickly scroll down
07:31
for some more of this stuff
07:34
and anyway you feel free to go do a
07:36
search
07:37
by yourself you'll find that these
07:38
things go way back
07:40
i mean they go way way way back when
07:44
twitter first started this
07:45
this tweet just kept getting bounced
07:48
around
07:48
ten times more effective indeed now
07:51
clearly there must be
07:52
like an origin to this really specific
07:56
claim like 10 times
07:58
and hopefully i'll figure some of that
08:00
out once we do our
08:01
actually get our hands dirty with the
08:03
research synthesis project
08:05
is my guess is there's probably is a
08:06
publication out there that makes that
08:08
claim
08:10
and and as i go through all the
08:13
published studies
08:14
you know we'll find it and we'll be like
08:16
aha this is the one
08:18
this is the one where the world has
08:21
decided that catnip is the best thing
08:24
and look at this look at this one right
08:25
here catnip is 10 times more effective
08:28
at repelling mosquitoes than deet
08:30
deet is commonly used in insect
08:32
repellents
08:34
but that's a dragonfly that's not a
08:36
mosquito anyway
08:38
that's twitter for you
08:41
okay so
08:46
so before starting the project let me
08:47
just zoom
08:49
really quickly into what i mean by
08:51
research synthesis
08:54
it is kind of like a specific tool set
08:58
um it is not the wild west
09:02
in terms of all the things that i'm
09:04
going to do are
09:06
have associated best practices
09:11
um but i'm i'm probably going to deviate
09:13
from a lot of that stuff
09:14
mostly because you know i want to get
09:15
the project done quickly
09:18
and so what i'm going to have is a um
09:22
a wall of shame i guess i don't know
09:24
what else to call it uh
09:25
deviations from best practices where
09:28
every time i
09:30
cut a corner or do something that's not
09:34
really standard protocol i'm going to
09:36
put it on the list
09:37
and you'll see that every um and this
09:40
list is going to get
09:42
huge i mean i'm probably going to
09:44
deviate from best practices
09:46
almost every decision i make just to
09:48
complete the project quickly
09:52
and so this is to give you an idea of
09:54
like okay
09:55
you know what mark you me you
09:57
potentially could have messed up
09:59
at any one of these stages and i'm going
10:02
to try to
10:03
discuss very briefly why
10:07
the decisions i make deviate from best
10:09
practices so that you know
10:11
when it comes for you to actually try to
10:13
complete one of these projects
10:15
then you shouldn't quite do what i'm
10:17
doing that you should spend more time to
10:19
think about certain
10:20
um details on and
10:24
how they could impact your synthesis
10:26
downstream
10:29
now uh synthesis i've kind of been
10:30
throwing this
10:32
word out a lot what do i mean by
10:35
research synthesis i really
10:36
mean something very specific and let me
10:40
jump into powerpoint here
10:41
and this will be the only powerpoint
10:45
stuff that i'm going to have for this
10:47
class and this is just recycled from
10:49
my other meta-analysis course
10:53
what i mean by research synthesis is
10:55
basically two branches
10:57
of um met two
11:00
methodologies in which scientists use to
11:04
combine and compare study outcomes
11:08
the one that i am most comfortable and
11:11
familiar with
11:12
is meta-analysis and
11:16
basically meta-analysis is like a
11:18
statistical tool
11:20
to combine and compare research outcomes
11:22
it's heavy on the stats
11:26
and if i have time i will talk about the
11:28
details
11:29
of how we statistically analyze the
11:32
outcomes of stats these
11:35
these models aren't intuitive
11:38
per se um but they do have a really
11:42
firm foundation in how we think
11:46
um studies vary and change
11:50
and how we can account for that
11:51
statistically
11:55
and so meta-analysis you could split up
11:56
into two
12:00
gears in terms of
12:04
what kind of mechanics are involved in
12:06
achieving
12:07
a meta-analysis statistically you want
12:10
to combine and compare a research
12:12
outcome
12:13
for us with the catnip studies i think
12:16
it's typically going to be a trial
12:20
where you have a
12:24
arm
12:27
compared to an another trial where you
12:30
have catnip on the arm
12:31
and you stick your arms in a cage and a
12:33
mosquito's land
12:36
on the arm and you count landing times
12:38
or probing times or feeding times on the
12:40
mosquitoes
12:41
as a quantitative measure of how
12:44
repellent
12:45
catnip is now the bear arm doesn't
12:48
necessarily need to be bare arm
12:51
that would be a negative control
12:54
because we're interested in like the
12:57
main claim
12:58
that catnip is 10 times more effective
13:01
than beet
13:02
ideally we would want deet as being the
13:05
positive control
13:06
the baseline for comparison that way we
13:08
could actually quantify whether or not
13:10
catnip is ten times more effective
13:12
indeed
13:14
another key aspect of meta-analysis is
13:16
like reporting
13:18
the stats reporting the aggregated
13:21
outcomes and so there's a key part of
13:23
visualizing and
13:24
uh reporting what you find
13:28
now before you even reach these stages
13:30
though
13:32
um there's a whole other branch of
13:35
research synthesis called systematic
13:36
review
13:38
and this is a collection of uh
13:41
best practices that are associated with
13:45
creating a repeatable robust
13:48
approach to finding studies
13:52
and so you begin with clear
13:56
vision of what you're looking for so
13:58
this is called scoping and searching
14:02
um and then when you find studies you
14:04
need to screen them
14:05
you need to find the full text
14:09
so that you could further screen them by
14:12
actually reading the study
14:14
and then you code details about each
14:17
study
14:18
and you pull out numerical values
14:20
important for meta-analysis
14:22
and so all these stages here typically
14:25
fall under
14:26
a systematic review and when
14:30
um the systematic review part is done
14:32
well
14:34
the meta-analysis part becomes that much
14:37
more
14:38
potent in terms of um how well the stats
14:42
work
14:44
and when they work together oh boy do we
14:47
get a wonderful thing
14:49
a high quality research synthesis
14:52
project
14:53
something that could provide
14:56
information across many many different
14:59
studies
15:01
perhaps reach a consensus um
15:04
on that information in our case reaching
15:06
a consensus of whether or not
15:08
catnip is more effective indeed
15:12
by again synthesizing what's been done
15:17
already
15:20
all right let me close this
15:25
okay so
15:28
let's start let's get our hands dirty i
15:30
mean i don't got
15:31
time to waste i've already talked
15:33
already for 15 minutes
15:35
let's get going so the goal now is this
15:37
phase which
15:38
i'm gonna call scoping i really don't
15:42
know what to expect
15:45
in terms of finding studies with catnip
15:47
i haven't done this before
15:49
i have an idea of repellency studies
15:52
i've certainly been part of many
15:53
projects on that
15:54
but nothing specific with cadnip and so
15:57
the scoping phase is for me to
16:01
understand what i need in order to find
16:05
all the studies that
16:08
use catnip as a repellent against
16:11
mosquitoes
16:12
that's my goal now am i going to find
16:14
all the studies probably not
16:15
again i'm going to cut corners because i
16:17
need to get this done quick
16:20
but i need to find um empirical
16:24
studies or it's something that does a
16:26
manipulative experiment
16:30
and so for me to do to at least find a
16:32
few
16:33
i need to get an idea of what to search
16:35
for
16:36
and i imagine catnip is not
16:40
not really the word to try to catch a
16:45
lot of these studies
16:46
you know scientific literature tend to
16:48
use more
16:49
scientific terms and i'm sure they would
16:52
probably use a species name of catnip
16:55
i'm sure catnip is probably like a
16:56
collection of species of plants
17:00
and so the scoping phase is for me to
17:02
figure out
17:03
those details so i could formulate
17:05
proper keywords
17:07
to start finding studies and so right
17:10
now i'm just going to kind of
17:11
do a quick wikipedia search
17:15
to try to find all the words associated
17:18
with catnip
17:21
so that when i do my search i could
17:22
catch those studies that kind of
17:24
use different word words
17:28
different ways to describe that one
17:30
plant
17:32
so let me get my paper here okay
17:35
so let's go on to
17:39
oops uh
17:42
firefox here i already got some
17:46
some stuff here all right i'm just gonna
17:50
do a search for catnip
17:59
wikipedia okay so my goals now again is
18:01
scoping is i'm trying to find the
18:03
keywords to try to find all the studies
18:05
that use catnip as a repellent
18:08
and while this loads
18:11
um i could already tell right now that
18:13
the species name of catnip
18:15
is uh nepeta
18:18
cateria i did not know that so that will
18:20
be part of my search terms
18:22
is i'll have catnip the common name and
18:26
include the species name or at least
18:27
maybe the genus
18:29
only so i got cat nip i'm just going to
18:32
write this down
18:33
on a piece of paper
18:39
nepeta i'm only going to use the genus
18:41
and then and then
18:43
um
18:49
and then there's also other common names
18:51
for catnip cats wart
18:53
catwart cat mint can't mint
18:57
i kind of like that
19:00
so i'm going to write those down
19:03
also because they will also become part
19:06
of my keyword search
19:09
okay wart
19:13
cat mint okay genus nepata
19:17
um
19:21
is there i need to know the compound now
19:24
with a lot of these repellency studies
19:26
they're not actually using leaves of
19:28
catnip they're using
19:29
a plant-derived compound likely an
19:32
essential oil
19:33
that was extracted from the plant
19:37
presumably the catnip has like a
19:41
secondary compound a chemical
19:44
that is the focus of that will be the
19:46
focus of all these trials
19:48
which is presumably the chemical that is
19:52
the active agent
19:53
as a repellent against mosquitoes so i
19:56
i'm just going to quickly zip through
19:58
here see if we could find
20:00
i mean why wouldn't that be in the
20:01
wikipedia page is this a chemical
20:05
there we go effects on cats catnip
20:08
contains the
20:09
feline attractant nepati lactone
20:13
okay so that's kind of a convenient uh
20:17
chemical name it has the actual genus of
20:19
the plant
20:20
in the name that's nice nepa
20:26
lactone all right so that's going to
20:28
become part of the
20:32
search which is the chemical and let me
20:34
click on the chemical here see what it
20:36
happens
20:38
i mean it's kind of funny right this is
20:40
like a bizarre thing
20:42
catnip may repel mosquitoes but it may
20:44
attract cats
20:46
it's you know this this weird wonderful
20:50
um diversity of
20:55
outcomes that could come with um just
20:58
this one chemical
21:01
organic compound isolated from catnip
21:04
which acts as a cat attractant
21:06
or mosquito repellent whatever our
21:10
findings
21:11
will be we could modify this wikipedia
21:13
page
21:14
with their results and then i'm not i'm
21:18
not really concerned about
21:19
what is the actual chemical uh
21:23
effect on that okay so i think i got
21:26
what i needed
21:26
i got the common names a collection of
21:28
common names
21:30
i got the genus of the plant and i got
21:33
the chemical
21:34
associated with it i think that would
21:37
encompass the broad diversity of stuff
21:42
needed to identify many studies now this
21:45
is where we're going to go
21:46
to our um wall
21:50
of deviations my first
21:53
real mess up so right now it says
21:55
synthesis currently perfect
21:57
but i'm going to mess all that up
21:59
because i did
22:01
let's be realistic here i kind of did a
22:03
quick and dirty job in formulating
22:05
keywords
22:07
and so the scoping phase you need to
22:10
spend more time than me than just
22:12
10 minutes trying to figure out how to
22:14
find studies
22:16
you need to read a collection of studies
22:20
to get a good idea of what the words are
22:23
used
22:24
what words what language semantic
22:27
language is associated with
22:29
um your project
22:33
and you do not want to just exclude a
22:35
bunch of potentially relevant studies
22:37
because you didn't quite
22:38
use the right terminology to
22:42
identify which are relevant so my
22:45
deviation
22:46
my first deviation here will be
22:52
that
22:54
in the scoping phase
22:58
um did
23:01
not read studies
23:06
right so i didn't i i don't think i've
23:08
ever read
23:09
a repellency study on catnip
23:13
right typically when you reach this
23:14
phase you've read a lot already about
23:16
the subject and you're ready to kind of
23:18
tackle it
23:19
um and you can you have an idea a kernel
23:22
in your head
23:23
on how to properly approach it i have no
23:25
clue
23:26
and so this is the first failure on my
23:29
part in the research synthesis
23:31
project is um i don't know much about
23:35
the literature
23:41
and so here you go
23:45
there you go first failure is i did a
23:48
really crummy job in scoping in
23:51
formulating keywords
23:55
okay so let's plug in some of these
23:57
keywords into
23:59
webassigns which is a
24:02
collection of sources to find
24:06
bibliographic information um
24:12
and let's try to find some studies
24:16
i mean i got some keywords
24:20
i already kind of did a search for beans
24:21
here to make to test whether or not
24:23
weber science was working
24:26
okay so what i want is cat
24:30
nip common name
24:34
and then i'm going to add a boolean here
24:35
of and so i want to be inclusive
24:38
of other ways to describe this plant
24:42
catnip cat swart
24:47
and cat
24:51
wart and my favorite
24:54
cat mint
24:59
and now i want to add the genus name
25:02
to the search term nepata
25:05
nepeta i'll figure it out i'll figure
25:09
out how to pronounce that at some point
25:11
and then finally is a chemical name
25:15
nepetalone
25:20
nepeta lactone sorry nepeta lactone
25:27
all right so here i got the common names
25:30
genus and chemical the plant derived
25:33
compound from a plant
25:34
this should be enough to find some
25:37
studies
25:38
and then finally um
25:43
oh what am i doing i'm not doing and
25:46
okay first mistake second mistake
25:48
already
25:54
this is to go under scoping again
25:58
um poor
26:02
formulation of keywords
26:12
right okay i already messed up i messed
26:15
up my booleans
26:16
and um it's like an
26:20
exclusion term right when the more ands
26:22
you add
26:23
the smaller the pool of studies you find
26:27
what i wanted what i meant here was or
26:31
the more ores you add the greater the
26:33
collection of studies
26:35
balloons and that's the goal you should
26:38
try to be
26:38
as inclusive as you can from the start
26:42
and then once you screen things once you
26:45
go through each
26:46
study that's when you start dropping
26:48
things
26:49
you goal is to have eyes on the studies
26:52
you making the decision on whether or
26:54
not to include exclude
26:56
not search not search terms
26:59
don't let the search terms be the
27:01
exclusion
27:03
uh the phase in which you exclude
27:05
studies
27:06
i is better than a database making those
27:09
decisions for you
27:12
okay so let's back to our search
27:16
replace all these search terms with or
27:31
and then finally now i want to narrow my
27:34
search i don't want all the studies on
27:36
catnip
27:37
i want to narrow my search to studies
27:39
that use catnip as a repellent
27:42
and so this is where i throw in my and
27:48
and uh repellent
27:52
is a fairly standard word you use to
27:56
describe this type of experiment so i'm
27:58
not too worried that i'm gonna
28:00
exclude or miss out on many studies
28:02
using
28:03
variations of the word but i will
28:06
add a wild card to the word repel which
28:09
will catch
28:10
most of these uh variations of like
28:13
repellent repellency
28:15
repel i mean
28:19
and the wild card is just little
28:20
asterisks
28:24
and that's it i'm not even gonna throw
28:27
in mosquitoes
28:29
in there um because you we may
28:32
find studies that use a collection of
28:35
organisms to test repellency effects
28:40
and so really all i want is catnips and
28:44
repel
28:45
and then hopefully i'm gonna get a ton
28:47
of ton of things to screen
28:49
put my eyes on and and then i will make
28:52
the decision on whether or not they're
28:54
relevant
28:55
right so if a study is on cockroaches
28:58
you know i'll know to exclude it um
29:01
i'll be comfortable with that exclusion
29:04
because i made the decision
29:06
i didn't let weber science make the
29:08
decision for me
29:09
so let's do a search for this
29:12
and see what comes up hopefully there's
29:15
going to be more than like five studies
29:19
oh thank you already an issue
29:28
oh i lost all that stuff
29:34
um all right let's try it again
29:39
luckily all that is oh that's because i
29:41
got a ton of brackets
29:46
and
29:51
yes missing parentheses
29:55
this is the problem with doing the oh
29:57
come on friends
30:02
sign in okay so this is the problem i'm
30:05
going to do a copy and paste here of
30:09
my search terms i'm actually going to
30:13
throw that in a word document because i
30:15
want to keep track of
30:16
all these decisions that i make
30:26
there we go these are my search terms
30:30
um i mean i might as well just start
30:32
writing the manuscript right now
30:38
uh on what day it is
30:42
this on december 2nd
30:45
2020
30:48
i searched
30:52
web of science
30:56
with these key words
31:03
there we go now
31:07
another deviation from best practice
31:09
let's go back to our list
31:14
i'm only doing uh two things i'm do only
31:17
searching web of science which is a
31:19
problem
31:26
you want to search multiple sources
31:30
of studies so i'm only limiting myself
31:33
to web of science but there are many
31:35
other
31:37
repositories of this information you
31:39
could hit on google scholar is a popular
31:41
one
31:44
i'm not gonna do any of that stuff um
31:47
mostly because i need to move on i've
31:48
already talked way too much
31:50
and so let me just add this to the wall
31:52
of shame here
31:54
searching only
31:57
searched one database
32:02
and i'm going to add another
32:06
one where
32:09
poorly kept
32:14
track of database
32:17
details web of science is
32:22
a goofy place to find studies
32:25
mostly because you don't really quite
32:27
know what databases they're searching
32:30
um but there's ways to sniff out
32:34
what exactly where
32:37
all these studies come from and so
32:40
what i'm gonna do here is uh while
32:46
let's uh save this
32:50
head over to the wall of shame and i
32:53
think what probably needs to happen is i
32:55
need to log
32:56
back into webassigns
33:04
go back to database now here's the thing
33:08
is um
33:11
it's super hard to do a research
33:12
synthesis project unless you have
33:15
institutional access to
33:18
databases and online
33:23
um e-journal access that is really the
33:26
super bummer
33:27
of these types of projects is if you do
33:30
not have
33:33
access to this stuff either through a
33:37
or through some a library
33:40
um you're really limited in what kind of
33:43
studies you could find not all
33:45
papers are available for the public to
33:48
read
33:49
they're behind paywalls and let me tell
33:52
you something if you're a
33:54
graduate student or an undergraduate
33:56
trying to do one of these projects never
33:58
pay
33:59
for a research article i mean who's
34:02
going to pay 100 bucks to look at a read
34:04
a nature paper that seems like the most
34:06
ridiculous thing ever
34:08
there are certainly more nefarious ways
34:10
to get those
34:12
papers um
34:15
cy hub yeah
34:19
okay all right so let's do back our
34:22
search i think i'm
34:23
logged back in
34:26
let's find out what these keywords
34:30
have found oh yeah a little bit about
34:32
the databases so here i'm
34:34
searching web science core collection
34:36
i'm going to go all databases
34:39
and if you ever wonder what all these
34:40
databases are you could just
34:43
it has to update itself there we go we
34:46
got our searches you go to
34:48
under more settings and it's going to
34:50
search from
34:51
whatever archives they have from 1864 to
34:55
2020 and here is the collection of
35:00
databases that they searched i mean i
35:02
could copy and paste this
35:06
and stick it in my word file
35:12
so that later when i come back to
35:13
actually writing the manuscript i'm
35:15
going to actually
35:16
have information on what databases the
35:19
web of science provided
35:23
okay i need to move on with this
35:26
so let's search for
35:30
with these keywords let's see what comes
35:32
up
35:36
oh please work
35:39
missing parentheses oh yeah okay i never
35:41
fixed that weird issue where there was a
35:42
bunch of
35:44
existing parentheses
35:49
there we go
35:54
all right fingers crossed that there's
35:55
more than two studies
35:58
244 i mean you can refine your search
36:02
right i did a huge search including
36:05
patents
36:06
maybe the patent literature is important
36:09
for repellency
36:11
um experiments but my experience
36:15
and my gut feeling is they tend to not
36:16
provide um
36:18
actual experimental results within a
36:20
patent
36:21
they just say hey here's a chemical
36:23
concoction that
36:24
we derived it has these properties
36:28
um they don't actually report that this
36:30
is uh
36:32
you know 2.5 times more effective than
36:36
than deet but i'm going to search these
36:39
and include these in my
36:40
search maybe there is a patent out there
36:44
that actually reports a numerical
36:45
outcome useful for synthesis
36:47
but anyway i got 244 with just the funny
36:50
search terms
36:52
and that i pulled together quickly and
36:54
let's quickly zip through some of these
36:56
and i can tell you right now some of the
36:58
top hits are just like totally nonsense
37:00
i mean drosophila repellency of
37:02
drosophila
37:05
evaluation of lotions of botanical based
37:08
repellents okay so this study probably
37:10
includes
37:11
a catnip not an ideal study because the
37:15
catnip is probably tested with a
37:17
collection of compounds this is not
37:19
unusual for repellency experiments
37:22
but here okay so repentancy of catnip
37:25
oils on bed bugs
37:27
not relevant and so okay there's there's
37:31
stuff out there
37:33
okay so the final step i'm going to
37:35
quickly get this done is just to
37:36
download these
37:40
you go to export to file format
37:43
webassigns is a bizarre
37:45
it only allows you to download 500
37:47
things at a time
37:50
i mean what can you do about that stuff
37:52
and i want abstract so i'm going to
37:54
include all that stuff
37:55
and i'm going i'm not going to use the
37:56
tab delimited utf-8
37:59
utf-8 you're going to get i think i
38:02
always get these backwards you're going
38:04
to get a whole bunch of weird characters
38:06
and you'll find that um because i'm
38:08
going to use r
38:09
a lot for the screening
38:12
and analysis part weird characters just
38:15
don't work well
38:16
r doesn't digest weird characters very
38:19
well
38:20
and so we want to avoid using
38:24
non-roman characters non-i don't know
38:27
what you call these things but anyway
38:29
you don't want a bunch of goofy
38:32
symbols messing up your
38:37
your references and so i'm just going to
38:39
use tab delimited for windows i think
38:41
that's not
38:42
you that might be a lesser format which
38:45
is just ascii code
38:47
and r loves ascii does not
38:50
love other things and so we're going to
38:53
export that
38:56
basically it's going to collate these 2
38:57
44 that's fairly
38:59
straightforward sending it to file
39:05
and it needs to think about that
39:09
i mean i guess i could open it in
39:10
notepad you're not going to be able to
39:12
see this
39:14
okay so there you go i saved it
39:18
i'm going to end today's lecture with
39:20
that
39:22
maybe provide a let me just quickly
39:28
um summarize what i did today what the
39:30
point of the course is
39:33
my goal is to synthesize
39:36
studies that use catnip as a repellent
39:38
to mosquitoes
39:41
i want to test this claim that's
39:44
floating out there where catnip is 10
39:47
times more effective than rep
39:48
than deet to me this is a bizarre claim
39:53
i feel like there's some sampling error
39:56
where a study just did not have adequate
39:58
sampling and
40:00
achieved this result i'll talk a bit
40:02
about that in
40:03
in the following lectures um
40:08
and so i did already two phases in
40:12
the systematic review synthesis part
40:14
where i
40:15
uh scoped the literature
40:19
not quite right i could just kind of
40:21
pull this out
40:22
fairly quickly i pulled together a bunch
40:24
of keywords
40:25
i'm interested in finding catnip studies
40:29
and then i searched web of science to
40:31
try to find
40:32
those studies i broke a lot of rules
40:34
already
40:36
right i i spent little time
40:39
formulating my keywords and i only
40:42
searched one database
40:44
so what you need to put in your back
40:46
pocket is
40:48
if you're going to start thinking about
40:50
a research synthesis project
40:52
read the literature beforehand i'm sure
40:56
right now there's probably a published
40:58
review of catnip as a chemical
41:01
as a repellency as an attractant which
41:03
would have been incredibly useful for me
41:05
to formulate
41:07
really precise keywords to identify
41:09
studies
41:10
didn't do that two i'm probably not
41:14
going to find the entire
41:16
domain of knowledge associated with
41:19
catnips as a repellent
41:21
because i've limited myself to a single
41:24
database web of science
41:28
you're probably going to want to hit on
41:29
multiple databases
41:33
to actually get a better collection
41:36
of studies to synthesize other databases
41:39
include
41:40
you know open access dissertations
41:45
google scholar although google saw
41:47
scholar really is like a
41:49
funny black box it's not really it like
41:51
a database search
41:53
and google tries to answer questions
41:56
that's the that's what its goal is and
41:59
when you do a search on google scholar
42:00
is trying to provide an answer for what
42:03
you're looking for
42:04
it's not giving you a straight up
42:07
conventional database output
42:11
of all the things associated with
42:14
that search criteria is trying to
42:16
provide an answer
42:18
to your query um
42:22
which means that it's gonna provide a
42:24
ton of stuff that's not relevant
42:27
but a ton of stuff that might be what it
42:30
thinks
42:30
is what you're looking for web of
42:33
science doesn't quite do that
42:35
it's actually used searching the
42:36
abstracts and the titles
42:38
and then it has its own formulated
42:40
keywords that it searches
42:42
so basically it's like okay i have
42:45
a million studies here are all the
42:48
studies with catnip
42:51
google scholar on the other hand
42:53
probably has like
42:54
10 million studies or more and it says
42:57
oh
42:58
you're looking for catnip here's what i
43:00
think
43:01
you'd be interested in which is kind of
43:03
a very different approach to
43:06
finding studies now the neat part about
43:07
google scholar is it actually searches
43:09
the full text of studies
43:11
and so you might catch a few odd eggs
43:15
that do report what you're looking for
43:18
but is not reported in the abstract or
43:22
title
43:23
of the study there are limitations there
43:26
are advantages
43:28
the compromise is just to hit many
43:31
different places
43:32
to try to find your studies now of
43:34
course you need to keep track
43:36
of all those searches that you make
43:39
in order to make it repeatable for other
43:42
scientists
43:43
to evaluate what you've done
43:46
repeatability is
43:47
crucial because uh
43:50
because again you could deviate from
43:52
best practices at any point
43:54
in a research synthesis project and that
43:56
might impact downstream
43:59
what you have in order to reach a
44:02
consensus
44:03
on multiple study outcomes
44:06
if you do deviate that's fine
44:10
as long as you're open to describing in
44:14
detail what you've done
44:16
right make things as transparent as
44:18
possible
44:19
okay so next lecture
44:22
we are going to do the super dull
44:26
part which is screen all these things i
44:28
downloaded from webassigns
44:31
include exclude studies again i'm just
44:33
going to quickly eyeball these things
44:35
for studies that
44:36
basically test on mosquitoes um
44:40
and then we will begin to try to
44:42
retrieve
44:43
those studies actually get the full text
44:46
so that we can start pulling out
44:47
numerical values for
44:50
meta-analysis anyway i've talked for too
44:54
long
44:57
take it easy