class: split-70 with-border hide-slide-number bg-brand-red background-image: url("images/USydLogo-black.svg") background-size: 200px background-position: 2% 90% .column.white[.content[ <br><br><br> # Motivating Examples ## .black[STAT3022 Applied Linear Models Lecture 21] <br><br><br> ### .black[2020/02/18] ]] .column.bg-brand-charcoal[.content.white[ ## Today 1. Exploratory Data Analysis 2. Data cleaning ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # The Avocado Lover's job problem ]] .row[.split-60[ .column.bg-gray.white[.content[ Dear statistician, I work in an Avocado company that has many offices in US. My boss asked me to analyse the US market for the price and sales trends and factors that drive these trends. I am able to obtain a data set to analyse but I need your help to advise me how to conduct this research. Personally, I am also an avocado lover and eat avocado toast nearly every day. My boss told me that he will send me to an US office in July 2019 to get some trainings and let me choose which office to work in. My salary is not too high so I need to make sure that avocado is least expensive there. I like organic avocado but if it's too costly, I'm willing to settle for non-organic avocado. Please help me decide where I should go! Kind regards, Avocado Lover ]] .column[.content.center.black.vmiddle[ # What is the aim? ]] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # The Avocado Lover's job problem ]] .row[.split-60[ .column.bg-gray.white[.content[ Dear statistician, I work in an Avocado company that has many offices in US. My boss asked me to analyse the US market for the .yellow[**price and sales trends and factors that drive these trends**]. I am able to obtain a data set to analyse but I need your help to advise me how to conduct this research. Personally, I am also an avocado lover and eat avocado toast nearly every day. My boss told me that he will send me to an US office in .yellow[**July 2019**] to get some trainings and let me choose which office to work in. My salary is not too high so I need to make sure that .yellow[**avocado is least expensive**] there. I .yellow[**like organic avocado**] but if it's too costly, I'm willing to .yellow[**settle for non-organic avocado**]. Please help me decide .yellow[**which state**] I should go! Kind regards, Avocado Lover ]] .column[.content[ First identify what the client wants. * Price and sales trends and their driving factors. * Personally, the price in **July 2019**. * Recommendation of which **state** to go. More beneficial is to identify city in which her workplace has an office. * The client cares moderately for **organic** avocados. How much is she willing to pay extra for it? * The client may also care about cost of living other than avocados (rent, electricity, water bills, etc), lifestyle, etc. ]] ]] --- layout: false class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Do you have the data to answer the question? ]] .row[.content[ * In real world, the questions are often not clearly defined. * Keep in mind that the client may find it hard to pinpoint exactly what they want. * It is your job as a statistician to work together with the client to frame the problems numerically/statistically. * Even after you form the questions analytically, you may have insufficient data to answer the questions. * You may need to make the best out of data, acknowledging the potential flaws in the analysis. * In some cases, you just simply cannot gain any information out of data. * "If you torture the data long enough, it will confess." .pull-right[ -Ronald H. Coase] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Understand avocado prices and sales volume in US ]] .row[.content[ ```r skimr::skim(dat) ``` <img src="images/L21Skim.png" width="80%" height="80%"> ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Get data summary ]] .row[.split-60[ .column[.content[ ## What does each variable mean? ```r dat <- dat %>% mutate(Date=as.Date(Date, format="%d/%m/%y")) %>% janitor::clean_names() # better names for analysis str(dat) ``` ``` tibble [18,249 x 13] (S3: spec_tbl_df/tbl_df/tbl/data.frame) $ date : Date[1:18249], format: "2015-12-27" "2015-12-20" "2015-12-13" ... $ average_price: num [1:18249] 1.33 1.35 0.93 1.08 1.28 1.26 0.99 0.98 1.02 1.07 ... $ total_volume : num [1:18249] 64237 54877 118220 78992 51040 ... $ x4046 : num [1:18249] 1037 674 795 1132 941 ... $ x4225 : num [1:18249] 54455 44639 109150 71976 43838 ... $ x4770 : num [1:18249] 48.2 58.3 130.5 72.6 75.8 ... $ total_bags : num [1:18249] 8697 9506 8145 5811 6184 ... $ small_bags : num [1:18249] 8604 9408 8042 5677 5986 ... $ large_bags : num [1:18249] 93.2 97.5 103.1 133.8 197.7 ... $ x_large_bags : num [1:18249] 0 0 0 0 0 0 0 0 0 0 ... $ type : chr [1:18249] "conventional" "conventional" "conventional" "conventional" ... $ year : num [1:18249] 2015 2015 2015 2015 2015 ... $ region : chr [1:18249] "Albany" "Albany" "Albany" "Albany" ... ``` ]] .column.bg-brand-gray.font_sm80[.content[ <br> ### Given description * `date` - the date of the observation * `average_price` - the average price of a single avocado * `type` - conventional or organic * `year` - the year * `region` - the city or region of the observation * `total_volume` - Total number of avocados sold * `x4046` - total number of avocados with PLU 4046 - small/medium Hass avocado (about 3-5oz avocado) - sold * `x4225` - total number of avocados with PLU 4225 - large Hass avocado (about 8-10oz avocado) - sold * `x4770` - total number of avocados with PLU 4770 - extra large Hass avocado (about 10-15oz avocado) - sold The PLU or Price Look-Up code is a 4- or 5-digit number that is primarily used on fresh produce items. ]] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Do your research of avocado market! ]] .row[.split-60[ .column[.content[ Original data source: http://www.hassavocadoboard.com/retail/volume-and-price-data <img src="images/kaggle_map.png" width="100%"> ]] .column[.content[ <br><br><br><br><br><br> <img src="images/kaggle_mapkey.png" width="100%"> ]] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Understand regions ]] .row[.content[ .font_sm80[ ```r unique(dat$region) ``` ``` [1] "Albany" "Atlanta" "BaltimoreWashington" "Boise" [5] "Boston" "BuffaloRochester" "California" "Charlotte" [9] "Chicago" "CincinnatiDayton" "Columbus" "DallasFtWorth" [13] "Denver" "Detroit" "GrandRapids" "GreatLakes" [17] "HarrisburgScranton" "HartfordSpringfield" "Houston" "Indianapolis" [21] "Jacksonville" "LasVegas" "LosAngeles" "Louisville" [25] "MiamiFtLauderdale" "Midsouth" "Nashville" "NewOrleansMobile" [29] "NewYork" "Northeast" "NorthernNewEngland" "Orlando" [33] "Philadelphia" "PhoenixTucson" "Pittsburgh" "Plains" [37] "Portland" "RaleighGreensboro" "RichmondNorfolk" "Roanoke" [41] "Sacramento" "SanDiego" "SanFrancisco" "Seattle" [45] "SouthCarolina" "SouthCentral" "Southeast" "Spokane" [49] "StLouis" "Syracuse" "Tampa" "TotalUS" [53] "West" "WestTexNewMexico" ``` ] .pull-left[ * Some regions are unclear: * Albany in OR, GA and NY * Columbus in IN and OH * Jacksonville in FL * Portland in ME and OR ] .pull-right[ * Some regions seem to be a combination, e.g. `BuffaloRochester`, `CincinnatiDayton`, etc. * How do we verify where it is? * Is it possible? ] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Information on US cities ]] .row[.split-60[ .column[.content[ <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-6-1.svg" style="display: block; margin: auto;" /> ]] .column.bg-brand-gray[.content[ Below omits the `ggrepel::geom_label_repel`. ```r library(maps) states <- map_data("state") ggplot(states) + geom_polygon(aes(long, lat, group = group), color = "white", fill="black") + coord_fixed(1.3) + theme_void() + guides(fill=FALSE) ``` ```r us.cities %>% select(name, pop, lat, long) %>% head() ``` ``` name pop lat long 1 Abilene TX 113888 32.45 -99.74 2 Akron OH 206634 41.08 -81.52 3 Alameda CA 70069 37.77 -122.26 4 Albany GA 75510 31.58 -84.18 5 Albany NY 93576 42.67 -73.80 6 Albany OR 45535 44.62 -123.09 ``` ]] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Duplicate city names ]] .row[.split-60[ .column[.content[ <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-9-1.svg" style="display: block; margin: auto;" /> ]] .column.bg-brand-gray[.content[ ```r us.cities %>% filter(grepl("Albany", name) | grepl("Columbus", name) | grepl("Jacksonville", name) | grepl("Portland", name)) %>% select(name, pop, lat, long) ``` ``` name pop lat long 1 Albany GA 75510 31.58 -84.18 2 Albany NY 93576 42.67 -73.80 3 Albany OR 45535 44.62 -123.09 4 Columbus GA 184900 32.51 -84.87 5 Columbus IN 39453 39.21 -85.91 6 Columbus OH 741677 39.99 -82.99 7 Jacksonville FL 809874 30.33 -81.66 8 Jacksonville NC 68201 34.76 -77.40 9 Portland ME 62882 43.66 -70.28 10 Portland OR 542751 45.54 -122.66 ``` .font_sm80[ * Albany should be in NY * Portland should be in OR * Columbus should be in IN or OH (GA far away) * Jacksonville should be in FL or NC ] ]] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # How to choose Columbus? Use US City Information ]] .row[.split-70[ .column.font_sm80[.content[ Getting key variables to merge with original data. ```r USkey <- us.cities %>% mutate(region=substr(name, 1, nchar(name) - 3)) %>% rename(state=country.etc) %>% mutate(region=gsub(" ", "", region)) %>% select(region, pop, lat, long, state) %>% rbind(tribble(~region, ~pop, ~lat, ~long, ~state, "BaltimoreWashington", 602658 + 548359, (39.30 + 38.91)/2, (-76.61 -77.02)/2, "MD+DC", "BuffaloRochester", 276762 + 209587, (42.89 + 43.17)/2, (-78.86-77.62)/2, "NY+NY", "California", 26729306, 35.42923, -119.3301, "CA", "CincinnatiDayton", 301561+157607, (39.14 + 39.78)/2, (-84.51-84.20)/2, "OH+OH", "DallasFtWorth", 1216543+633849, (32.79+32.75)/2, (-96.77-97.34)/2, "TX+TX", "district of columbia", 548359, 38.91, -77.02, "DC", "GreatLakes", 15182052, 41.8975, -86.31265, "WI+IA+IL+MI+OH", "HarrisburgScranton", 47576 + 72516, (40.28+41.40)/2, (-76.88-75.67)/2, "PA+PA", "HartfordSpringfield",123836 + 152095, (41.77 + 42.12)/2, (-72.68-72.54)/2, "CT+MA", "MiamiFtLauderdale", 173597 + 386740, (26.14+25.78)/2, (-80.14-80.21)/2, "FL+FL", "Midsouth", 10669181, 37.51253, -79.53505, "KY+WV+TN+VA+NC+WV+MD+DE+DC", "NewOrleansMobile", 188467+454207, (30.68+30.07)/2, (-88.09-89.93)/2, "AL+LA", "Northeast", 21530612, 41.54582, -73.37044, "NJ+PA+NY+CT+RI+MA+VT+NH+ME", "NorthernNewEngland", 372834, 43.39, -71.14, "ME+NH+VT", # guess "PhoenixTucson", 1450884+525268, (33.54+32.20)/2, (-112.07-110.89)/2, "AZ+AZ", "Plains", 6719966, 41.87682, -94.22152, "ND+SD+NE+KS+MN+IA+MO", "RaleighGreensboro", 233342+350822, (36.08+35.82)/2, (-79.83-78.66)/2, "NC+NC", "RichmondNorfolk", 248182+189498, (36.92+37.53)/2, (-76.24-77.47)/2, "VA+VA", "SouthCarolina", 526667, 33.75429, -80.65286, "SC", "SouthCentral", 16433060, 31.9431, -96.3638, "TX+OK+LA+AR", "Southeast", 11204354, 29.29783, -82.32278, "MS+AL+GA+SC+FL", "StLouis", 315546, 38.64, -90.24, "MO", "TotalUS", 126175816, 37.49531, -95.10346, "AL+AK+AZ+AR+CA+CO+CT+DE+FL+GA+HI+ID+IL+IN+IA+KS+KY+LA+ME+MD+MA+MI+MN+MS+MO+MT+NE+NV+NH+NJ+NM+NY+NC+ND+OH+OK+OR+PA+RI+SC+SD+TN+TX+UT+VT+VA+WA+WV+WI+WY", "West", 15735863, 40.74418, -113.9845, "WA+OR+NV+ID+MT+WY+UT+CO+NM+AZ", "WestTexNewMexico",13721930, 31.44962, -98.13192, "TX+NM" # includes east Texas too )) ``` ]] .column.bg-brand-gray[.content[ How to choose? .brand-blue[IN or OH differ in pop size!] * Use total volume to predict pop size. * Add pop size information. * Some regions encompasses a large region. * Some regions are subset of other region. * Some coded here involve guess work and may have room for error. ]] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Is there a relationship with volume and population? ]] .row[.split-50[ .column[.content[ ```r us.cities %>% filter(grepl("Columbus", name)) %>% # IN or OH select(name, pop, lat, long) ``` ``` name pop lat long 1 Columbus GA 184900 32.51 -84.87 2 Columbus IN 39453 39.21 -85.91 3 Columbus OH 741677 39.99 -82.99 ``` ```r dat2 <- dat %>% left_join(USkey, by="region") %>% # leave these out for now filter(region!="Columbus") %>% filter(!grepl("+", state, fixed=T)) #drop comb ``` ```r dim(dat2) #dat 18429,13 drop columbus & comb ``` ``` [1] 12168 17 ``` ]] .column.bg-brand-gray[.content[ ```r library(lubridate) dat3 <- dat2 %>% filter(year==2017 & month(date)==7) ggplot(dat3, aes(total_volume, pop, color=type, group=type)) + geom_point() + scale_x_log10() + scale_y_log10() + geom_smooth(method="lm", se=FALSE, color="black") + theme(legend.position = "bottom") ``` <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-16-1.svg" style="display: block; margin: auto;" /> ]] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Predict the population of Columbus ]] .row[.split-50[ .column[.content[ ```r predat <- dat %>% filter(grepl("Columbus", region)) %>% filter(year==2017 & month(date)==7) %>% group_by(region, type) %>% summarise(total_volume=mean(total_volume)) %>% ungroup() predat #use tot vol at 7-2017 to pred pop size ``` ``` # A tibble: 2 x 3 region type total_volume <chr> <chr> <dbl> 1 Columbus conventional 195168. 2 Columbus organic 9886. ``` ```r M0 <- lm(log10(pop)~type+log10(total_volume),dat3) 10^predict(M0,newdata=predat, interval="prediction") %>% cbind(predat) ``` ``` fit lwr upr region type total_volume 1 282664.3 51813.66 1542047 Columbus conventional 195167.726 2 430889.5 79026.84 2349401 Columbus organic 9886.492 ``` ]] .column.bg-brand-gray[.content[ * Since the predicted population size is large, .brand-blue[Columbus appears to be in OH]. * Let's modify our key: ```r USkey_ncr <- USkey %>% filter(!(region=="Columbus" & state!="OH") & region %in% dat$region) dat_ncr<-dat %>% left_join(USkey_ncr,by="region") %>% filter(!grepl("+", state, fixed=T)) #no comb region dat_cr <- dat %>% left_join(USkey_ncr, by="region") ``` ```r dim(dat_ncr) ``` ``` [1] 12506 17 ``` ```r dim(dat_cr) ``` ``` [1] 19601 17 ``` ]] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Data coverage for her company offices? ]] .row[.split-60[ .column[.content[ <img src="images/avocadomap.png" width="100%"/> ]] .column.bg-brand-gray[.content[ * Some states are not well covered. ```r strsplit(dat_cr$state, "+", fixed=T) %>% unlist() %>% unique() ``` ``` [1] "GA" "NY" "OR" "MD" "DC" "ID" "MA" [8] "CA" "NC" "IL" "OH" "TX" "CO" "MI" [15] "WI" "IA" "PA" "CT" "IN" "FL" "NV" [22] "KY" "WV" "TN" "VA" "DE" "AL" "LA" [29] "NJ" "RI" "VT" "NH" "ME" "AZ" "ND" [36] "SD" "NE" "KS" "MN" "MO" "WA" "SC" [43] "OK" "AR" "MS" "AK" "HI" "MT" "NM" [50] "UT" "WY" ``` ]] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Boss's question: does sales (total) volume change? ]] .row[.content[ ```r julyperiod <- data.frame(xmin=as.Date(c("01/07/15", "01/07/16", "01/07/17"), format="%d/%m/%y"), xmax=as.Date(c("31/07/15", "31/07/16", "31/07/17"), format="%d/%m/%y"), ymin=min(dat$total_volume), ymax=max(dat$total_volume)) ggplot(dat_ncr) + geom_line(aes(date, total_volume, group=interaction(region,type), color=type)) + scale_y_log10() + geom_rect(data=julyperiod, mapping=aes(xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax), alpha=0.2) + guides(color=FALSE) ``` <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-23-1.svg" style="display: block; margin: auto;" /> ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Is the change of total volume over time significant? ]] .row[.split-50[ .column[.content[ .scroll-box-20[ ```r M1<-lm(total_volume~region:type+date,data=dat_ncr) summary(M1) #date has associated numerical value ``` ``` Call: lm(formula = total_volume ~ region:type + date, data = dat_ncr) Residuals: Min 1Q Median 3Q -2800515 -14931 -1542 9843 Max 5270823 Coefficients: (1 not defined because of singularities) Estimate (Intercept) -4.171e+05 date 2.475e+01 regionAlbany:typeconventional 8.866e+04 regionAtlanta:typeconventional 5.086e+05 regionBoise:typeconventional 7.861e+04 regionBoston:typeconventional 5.573e+05 regionCalifornia:typeconventional 5.934e+06 regionCharlotte:typeconventional 1.991e+05 regionChicago:typeconventional 7.556e+05 regionColumbus:typeconventional 1.650e+05 regionDenver:typeconventional 7.817e+05 regionDetroit:typeconventional 3.552e+05 regionGrandRapids:typeconventional 1.711e+05 regionHouston:typeconventional 1.174e+06 regionIndianapolis:typeconventional 1.700e+05 regionJacksonville:typeconventional 1.630e+05 regionLasVegas:typeconventional 3.080e+05 regionLosAngeles:typeconventional 2.922e+06 regionLouisville:typeconventional 8.810e+04 regionNashville:typeconventional 1.984e+05 regionNewYork:typeconventional 1.367e+06 regionOrlando:typeconventional 3.376e+05 regionPhiladelphia:typeconventional 4.069e+05 regionPittsburgh:typeconventional 1.013e+05 regionPortland:typeconventional 6.173e+05 regionRoanoke:typeconventional 1.357e+05 regionSacramento:typeconventional 4.325e+05 regionSanDiego:typeconventional 5.129e+05 regionSanFrancisco:typeconventional 7.771e+05 regionSeattle:typeconventional 5.984e+05 regionSouthCarolina:typeconventional 3.463e+05 regionSpokane:typeconventional 8.438e+04 regionStLouis:typeconventional 1.792e+05 regionSyracuse:typeconventional 5.784e+04 regionTampa:typeconventional 3.821e+05 regionAlbany:typeorganic -2.066e+03 regionAtlanta:typeorganic 7.263e+03 regionBoise:typeorganic -1.797e+03 regionBoston:typeorganic 9.806e+03 regionCalifornia:typeorganic 1.460e+05 regionCharlotte:typeorganic 2.819e+03 regionChicago:typeorganic 2.708e+04 regionColumbus:typeorganic 3.970e+03 regionDenver:typeorganic 3.175e+04 regionDetroit:typeorganic 1.159e+04 regionGrandRapids:typeorganic -8.433e+02 regionHouston:typeorganic 1.922e+04 regionIndianapolis:typeorganic 5.598e+02 regionJacksonville:typeorganic -1.138e+03 regionLasVegas:typeorganic 5.310e+03 regionLosAngeles:typeorganic 7.497e+04 regionLouisville:typeorganic -1.326e+03 regionNashville:typeorganic 3.861e+03 regionNewYork:typeorganic 4.896e+04 regionOrlando:typeorganic 9.964e+02 regionPhiladelphia:typeorganic 9.721e+03 regionPittsburgh:typeorganic 1.528e+03 regionPortland:typeorganic 2.837e+04 regionRoanoke:typeorganic 3.982e+03 regionSacramento:typeorganic 3.756e+03 regionSanDiego:typeorganic 9.938e+03 regionSanFrancisco:typeorganic 1.816e+04 regionSeattle:typeorganic 3.933e+04 regionSouthCarolina:typeorganic 4.669e+03 regionSpokane:typeorganic -7.563e+02 regionStLouis:typeorganic 2.143e+03 regionSyracuse:typeorganic -1.572e+03 regionTampa:typeorganic NA Std. Error (Intercept) 7.821e+04 date 4.527e+00 regionAlbany:typeconventional 1.535e+04 regionAtlanta:typeconventional 1.881e+04 regionBoise:typeconventional 1.881e+04 regionBoston:typeconventional 1.881e+04 regionCalifornia:typeconventional 1.881e+04 regionCharlotte:typeconventional 1.881e+04 regionChicago:typeconventional 1.881e+04 regionColumbus:typeconventional 1.881e+04 regionDenver:typeconventional 1.881e+04 regionDetroit:typeconventional 1.881e+04 regionGrandRapids:typeconventional 1.881e+04 regionHouston:typeconventional 1.881e+04 regionIndianapolis:typeconventional 1.881e+04 regionJacksonville:typeconventional 1.629e+04 regionLasVegas:typeconventional 1.881e+04 regionLosAngeles:typeconventional 1.881e+04 regionLouisville:typeconventional 1.881e+04 regionNashville:typeconventional 1.881e+04 regionNewYork:typeconventional 1.881e+04 regionOrlando:typeconventional 1.881e+04 regionPhiladelphia:typeconventional 1.881e+04 regionPittsburgh:typeconventional 1.881e+04 regionPortland:typeconventional 1.629e+04 regionRoanoke:typeconventional 1.881e+04 regionSacramento:typeconventional 1.881e+04 regionSanDiego:typeconventional 1.881e+04 regionSanFrancisco:typeconventional 1.881e+04 regionSeattle:typeconventional 1.881e+04 regionSouthCarolina:typeconventional 1.881e+04 regionSpokane:typeconventional 1.881e+04 regionStLouis:typeconventional 1.881e+04 regionSyracuse:typeconventional 1.881e+04 regionTampa:typeconventional 1.881e+04 regionAlbany:typeorganic 1.535e+04 regionAtlanta:typeorganic 1.881e+04 regionBoise:typeorganic 1.881e+04 regionBoston:typeorganic 1.881e+04 regionCalifornia:typeorganic 1.881e+04 regionCharlotte:typeorganic 1.881e+04 regionChicago:typeorganic 1.881e+04 regionColumbus:typeorganic 1.881e+04 regionDenver:typeorganic 1.881e+04 regionDetroit:typeorganic 1.881e+04 regionGrandRapids:typeorganic 1.881e+04 regionHouston:typeorganic 1.881e+04 regionIndianapolis:typeorganic 1.881e+04 regionJacksonville:typeorganic 1.629e+04 regionLasVegas:typeorganic 1.881e+04 regionLosAngeles:typeorganic 1.881e+04 regionLouisville:typeorganic 1.881e+04 regionNashville:typeorganic 1.881e+04 regionNewYork:typeorganic 1.881e+04 regionOrlando:typeorganic 1.881e+04 regionPhiladelphia:typeorganic 1.881e+04 regionPittsburgh:typeorganic 1.881e+04 regionPortland:typeorganic 1.629e+04 regionRoanoke:typeorganic 1.881e+04 regionSacramento:typeorganic 1.881e+04 regionSanDiego:typeorganic 1.881e+04 regionSanFrancisco:typeorganic 1.881e+04 regionSeattle:typeorganic 1.881e+04 regionSouthCarolina:typeorganic 1.881e+04 regionSpokane:typeorganic 1.881e+04 regionStLouis:typeorganic 1.881e+04 regionSyracuse:typeorganic 1.881e+04 regionTampa:typeorganic NA t value (Intercept) -5.333 date 5.467 regionAlbany:typeconventional 5.774 regionAtlanta:typeconventional 27.043 regionBoise:typeconventional 4.180 regionBoston:typeconventional 29.635 regionCalifornia:typeconventional 315.557 regionCharlotte:typeconventional 10.587 regionChicago:typeconventional 40.179 regionColumbus:typeconventional 8.776 regionDenver:typeconventional 41.567 regionDetroit:typeconventional 18.889 regionGrandRapids:typeconventional 9.100 regionHouston:typeconventional 62.454 regionIndianapolis:typeconventional 9.042 regionJacksonville:typeconventional 10.010 regionLasVegas:typeconventional 16.377 regionLosAngeles:typeconventional 155.373 regionLouisville:typeconventional 4.685 regionNashville:typeconventional 10.549 regionNewYork:typeconventional 72.693 regionOrlando:typeconventional 17.954 regionPhiladelphia:typeconventional 21.636 regionPittsburgh:typeconventional 5.385 regionPortland:typeconventional 37.904 regionRoanoke:typeconventional 7.217 regionSacramento:typeconventional 23.000 regionSanDiego:typeconventional 27.274 regionSanFrancisco:typeconventional 41.322 regionSeattle:typeconventional 31.822 regionSouthCarolina:typeconventional 18.417 regionSpokane:typeconventional 4.487 regionStLouis:typeconventional 9.527 regionSyracuse:typeconventional 3.076 regionTampa:typeconventional 20.318 regionAlbany:typeorganic -0.135 regionAtlanta:typeorganic 0.386 regionBoise:typeorganic -0.096 regionBoston:typeorganic 0.521 regionCalifornia:typeorganic 7.762 regionCharlotte:typeorganic 0.150 regionChicago:typeorganic 1.440 regionColumbus:typeorganic 0.211 regionDenver:typeorganic 1.688 regionDetroit:typeorganic 0.616 regionGrandRapids:typeorganic -0.045 regionHouston:typeorganic 1.022 regionIndianapolis:typeorganic 0.030 regionJacksonville:typeorganic -0.070 regionLasVegas:typeorganic 0.282 regionLosAngeles:typeorganic 3.987 regionLouisville:typeorganic -0.071 regionNashville:typeorganic 0.205 regionNewYork:typeorganic 2.604 regionOrlando:typeorganic 0.053 regionPhiladelphia:typeorganic 0.517 regionPittsburgh:typeorganic 0.081 regionPortland:typeorganic 1.742 regionRoanoke:typeorganic 0.212 regionSacramento:typeorganic 0.200 regionSanDiego:typeorganic 0.528 regionSanFrancisco:typeorganic 0.966 regionSeattle:typeorganic 2.092 regionSouthCarolina:typeorganic 0.248 regionSpokane:typeorganic -0.040 regionStLouis:typeorganic 0.114 regionSyracuse:typeorganic -0.084 regionTampa:typeorganic NA Pr(>|t|) (Intercept) 9.84e-08 date 4.68e-08 regionAlbany:typeconventional 7.90e-09 regionAtlanta:typeconventional < 2e-16 regionBoise:typeconventional 2.94e-05 regionBoston:typeconventional < 2e-16 regionCalifornia:typeconventional < 2e-16 regionCharlotte:typeconventional < 2e-16 regionChicago:typeconventional < 2e-16 regionColumbus:typeconventional < 2e-16 regionDenver:typeconventional < 2e-16 regionDetroit:typeconventional < 2e-16 regionGrandRapids:typeconventional < 2e-16 regionHouston:typeconventional < 2e-16 regionIndianapolis:typeconventional < 2e-16 regionJacksonville:typeconventional < 2e-16 regionLasVegas:typeconventional < 2e-16 regionLosAngeles:typeconventional < 2e-16 regionLouisville:typeconventional 2.83e-06 regionNashville:typeconventional < 2e-16 regionNewYork:typeconventional < 2e-16 regionOrlando:typeconventional < 2e-16 regionPhiladelphia:typeconventional < 2e-16 regionPittsburgh:typeconventional 7.36e-08 regionPortland:typeconventional < 2e-16 regionRoanoke:typeconventional 5.62e-13 regionSacramento:typeconventional < 2e-16 regionSanDiego:typeconventional < 2e-16 regionSanFrancisco:typeconventional < 2e-16 regionSeattle:typeconventional < 2e-16 regionSouthCarolina:typeconventional < 2e-16 regionSpokane:typeconventional 7.28e-06 regionStLouis:typeconventional < 2e-16 regionSyracuse:typeconventional 0.00210 regionTampa:typeconventional < 2e-16 regionAlbany:typeorganic 0.89299 regionAtlanta:typeorganic 0.69934 regionBoise:typeorganic 0.92389 regionBoston:typeorganic 0.60205 regionCalifornia:typeorganic 9.01e-15 regionCharlotte:typeorganic 0.88085 regionChicago:typeorganic 0.14984 regionColumbus:typeorganic 0.83282 regionDenver:typeorganic 0.09136 regionDetroit:typeorganic 0.53762 regionGrandRapids:typeorganic 0.96423 regionHouston:typeorganic 0.30671 regionIndianapolis:typeorganic 0.97625 regionJacksonville:typeorganic 0.94430 regionLasVegas:typeorganic 0.77767 regionLosAngeles:typeorganic 6.74e-05 regionLouisville:typeorganic 0.94379 regionNashville:typeorganic 0.83735 regionNewYork:typeorganic 0.00924 regionOrlando:typeorganic 0.95775 regionPhiladelphia:typeorganic 0.60522 regionPittsburgh:typeorganic 0.93526 regionPortland:typeorganic 0.08152 regionRoanoke:typeorganic 0.83230 regionSacramento:typeorganic 0.84168 regionSanDiego:typeorganic 0.59717 regionSanFrancisco:typeorganic 0.33413 regionSeattle:typeorganic 0.03650 regionSouthCarolina:typeorganic 0.80393 regionSpokane:typeorganic 0.96792 regionStLouis:typeorganic 0.90927 regionSyracuse:typeorganic 0.93340 regionTampa:typeorganic NA (Intercept) *** date *** regionAlbany:typeconventional *** regionAtlanta:typeconventional *** regionBoise:typeconventional *** regionBoston:typeconventional *** regionCalifornia:typeconventional *** regionCharlotte:typeconventional *** regionChicago:typeconventional *** regionColumbus:typeconventional *** regionDenver:typeconventional *** regionDetroit:typeconventional *** regionGrandRapids:typeconventional *** regionHouston:typeconventional *** regionIndianapolis:typeconventional *** regionJacksonville:typeconventional *** regionLasVegas:typeconventional *** regionLosAngeles:typeconventional *** regionLouisville:typeconventional *** regionNashville:typeconventional *** regionNewYork:typeconventional *** regionOrlando:typeconventional *** regionPhiladelphia:typeconventional *** regionPittsburgh:typeconventional *** regionPortland:typeconventional *** regionRoanoke:typeconventional *** regionSacramento:typeconventional *** regionSanDiego:typeconventional *** regionSanFrancisco:typeconventional *** regionSeattle:typeconventional *** regionSouthCarolina:typeconventional *** regionSpokane:typeconventional *** regionStLouis:typeconventional *** regionSyracuse:typeconventional ** regionTampa:typeconventional *** regionAlbany:typeorganic regionAtlanta:typeorganic regionBoise:typeorganic regionBoston:typeorganic regionCalifornia:typeorganic *** regionCharlotte:typeorganic regionChicago:typeorganic regionColumbus:typeorganic regionDenver:typeorganic . regionDetroit:typeorganic regionGrandRapids:typeorganic regionHouston:typeorganic regionIndianapolis:typeorganic regionJacksonville:typeorganic regionLasVegas:typeorganic regionLosAngeles:typeorganic *** regionLouisville:typeorganic regionNashville:typeorganic regionNewYork:typeorganic ** regionOrlando:typeorganic regionPhiladelphia:typeorganic regionPittsburgh:typeorganic regionPortland:typeorganic . regionRoanoke:typeorganic regionSacramento:typeorganic regionSanDiego:typeorganic regionSanFrancisco:typeorganic regionSeattle:typeorganic * regionSouthCarolina:typeorganic regionSpokane:typeorganic regionStLouis:typeorganic regionSyracuse:typeorganic regionTampa:typeorganic --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual standard error: 172900 on 12439 degrees of freedom Multiple R-squared: 0.9536, Adjusted R-squared: 0.9534 F-statistic: 3878 on 66 and 12439 DF, p-value: < 2.2e-16 ``` ] ``` [1] "2015-12-27" "2015-05-10" ``` ``` [1] 16796 16565 ``` ]] .column.bg-brand-gray[.content[ ```r M2 <- lm(total_volume ~ region:type, data=dat_ncr) anova(M2, M1) ``` ``` Analysis of Variance Table Model 1: total_volume ~ region:type Model 2: total_volume ~ region:type + date Res.Df RSS Df Sum of Sq 1 12440 3.7261e+14 2 12439 3.7172e+14 1 8.9302e+11 F Pr(>F) 1 2 29.884 4.676e-08 *** --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ``` * The overall `date` effect is significant partly due to the large sample size. * Since the `date` coefficient is +ve, `total_volume` increases over time in general. ]] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Model diagnostics? ]] .row[.split-70[ .column[.content[ ```r library(ggfortify) autoplot(M1) ``` <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-27-1.svg" style="display: block; margin: auto;" /> ]] .column.bg-brand-gray[.content[ * The reality is that the model diagnostics are not satisfied and thus inferences may not be reliable. * However outliers often present in large data set. * It is common to see an increasing trend of variance for a time series. ]] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Boss's question: Does price change with supply? ]] .row[.content[ ```r ggplot(dat_ncr) + geom_line(aes(total_volume, average_price, group=interaction(region,type), color=type)) + scale_y_log10() + scale_x_log10() + guides(color=FALSE) + geom_smooth(method="lm", se=FALSE, aes(total_volume, average_price, group=interaction(region,type)), color="black") #no overall line ``` <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-28-1.svg" style="display: block; margin: auto;" /> ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Does price change with supply? Overall trend? ]] .row[.content[ ```r ggplot(dat_ncr) + geom_line(aes(total_volume, average_price, group=interaction(region,type), color=type)) + scale_y_log10() + scale_x_log10() + guides(color=FALSE) + geom_smooth(method="lm", se=FALSE, aes(total_volume, average_price), color="black") #drop lines of each region & type ``` <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-29-1.svg" style="display: block; margin: auto;" /> ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Boss's question: Does price change over time? ]] .row[.content[ ```r julyperiod <- data.frame(xmin=as.Date(c("01/07/15", "01/07/16", "01/07/17"), format="%d/%m/%y"), xmax=as.Date(c("31/07/15", "31/07/16", "31/07/17"), format="%d/%m/%y"), ymin=min(dat$average_price), ymax=max(dat$average_price)) #July highlight ggplot(dat_ncr) + geom_line(aes(date, average_price, group=interaction(region,type), color=type)) + scale_y_log10() + geom_rect(data=julyperiod, mapping=aes(xmin=xmin,xmax=xmax,ymin=ymin,ymax=ymax), alpha=0.2) + guides(color=FALSE) #log scale ``` <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-30-1.svg" style="display: block; margin: auto;" /> ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Does price over time by region and type? ]] .row[.content[ <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-31-1.svg" style="display: block; margin: auto;" /> ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Does price change over time by region and type? ]] .row[.content[ <br> <img src="images/average_price_over_time.gif" width="100%"/> <br><br> ## Do we need data for other days beside July? ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Price in cities across July weeks each year ]] .row[.content[ <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-33-1.svg" style="display: block; margin: auto;" /> ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Price for each city across July weeks each year ]] .row[.content[ <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-34-1.svg" style="display: block; margin: auto;" /> ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Price for each combined region across July weeks ]] .row[.content[ <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-35-1.svg" style="display: block; margin: auto;" /> ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Mean price for region by type by year via modelling ]] .row[.split-40[ .column.bg-brand-gray[.content[ ```r julydat_ncr <- julydat_ncr %>% mutate(year=factor(year)) *M0 <- lm(average_price ~ *region:type:year-1,data=julydat_ncr) outM0 <- broom::tidy(M0) %>% arrange(estimate) %>% select(term, estimate) %>% separate(term, c("region", "type", "year")) %>% mutate(region=gsub("region", "", region), type=gsub("type", "", type), year=gsub("year", "", year)) %>% mutate(tyear= paste0(substring(type, 1, 1), year)) %>% select(region, tyear, estimate) %>% spread(tyear, estimate) outM0 ``` ``` # A tibble: 33 x 7 region c2015 c2016 c2017 o2015 o2016 <chr> <dbl> <dbl> <dbl> <dbl> <dbl> 1 Albany 1.19 1.38 1.51 2.03 1.49 2 Atlanta 1.05 0.936 1.23 1.49 1.34 3 Boise 1.09 0.748 1.34 1.94 1.48 4 Boston 1.22 1.48 1.55 2.10 1.52 5 Califo~ 1.14 1.15 1.38 1.82 1.56 6 Charlo~ 1.19 1.37 1.37 2.10 1.67 7 Chicago 1.21 1.52 1.55 1.65 1.89 8 Columb~ 1.02 1.11 1.15 1.27 1.24 9 Denver 1.16 1.09 1.22 1.43 1.41 10 Detroit 1.03 1.17 1.25 1.25 1.21 # ... with 23 more rows, and 1 more # variable: o2017 <dbl> ``` ]] .column.bg-white[.content[
]] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Mean price for region by type by year via means ]] .row[.split-40[ .column.bg-brand-gray[.content[ ```r outsum <- julydat_ncr %>% group_by(region, type, year) %>% *summarise(estimate=mean(average_price))%>% ungroup() %>% mutate(tyear= paste0(substring(type, 1, 1), year)) %>% select(region, tyear, estimate) %>% spread(tyear, estimate) outsum ``` ``` # A tibble: 33 x 7 region c2015 c2016 c2017 o2015 o2016 <chr> <dbl> <dbl> <dbl> <dbl> <dbl> 1 Albany 1.19 1.38 1.51 2.04 1.49 2 Atlanta 1.05 0.936 1.23 1.49 1.34 3 Boise 1.09 0.748 1.34 1.94 1.48 4 Boston 1.22 1.48 1.55 2.10 1.52 5 Califo~ 1.14 1.15 1.38 1.82 1.56 6 Charlo~ 1.19 1.37 1.37 2.10 1.67 7 Chicago 1.22 1.52 1.55 1.65 1.89 8 Columb~ 1.02 1.11 1.15 1.27 1.24 9 Denver 1.16 1.09 1.22 1.44 1.41 10 Detroit 1.03 1.17 1.25 1.25 1.21 # ... with 23 more rows, and 1 more # variable: o2017 <dbl> ``` ]] .column.bg-white[.content[
]] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Factors for price trend: Interaction effects for July ]] .row[.split-70[ .column[.content[ ```r gather(outsum, "typeyear", "estimate", -region) %>% mutate(type=case_when( substring(typeyear, 1, 1)=="c" ~ "conventional", substring(typeyear, 1, 1)=="o" ~ "organic"), year=substring(typeyear, 2, nchar(typeyear))) %>% ggplot(aes(year, estimate, group=region, color=region)) + facet_wrap(~type) + geom_point(size=2) + geom_line() + guides(color=FALSE) + scale_y_log10() + labs(x="year (July only)") ``` <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-40-1.svg" style="display: block; margin: auto;" /> ]] .column.bg-brand-gray[.content[ * We already know that `type` is an important factor for determining the `average_price`. * Is `year` or `date` important? * Is `year:region` important? * Is `year:type` important? * Is `year:type:region`? * What about `date:region:type`? ]] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Interaction effects ]] .row[.split-50[ .column[.content[ <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-41-1.svg" style="display: block; margin: auto;" /> <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-42-1.svg" style="display: block; margin: auto;" /> ]] .column.bg-brand-gray[.content[ * Is `year` or `date` important? * Yes! Early-year price appears to dip across all `region` and `type`. * Is `year:region` important? * This can be thought of as average of `year` by `region` combination. At specific dates, it seemed to behave differently but averaged over year may be not much. * Is `year:type` important? Maybe. * Is `year:type:region`? Maybe. * What about `date:type:region`? ]] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Testing for interaction effects ]] .row[.split-50[ .column[.content[ * Start with testing higher order interaction term `year:region:type`. * If a higher order interaction term is included, you cannot drop main effects (`year`, `region`, `type`) or lower order interaction term (e.g. `year:region`, `year:type`, `region:type`). * Because of the large data size, all terms including the higher order interaction are significant. ]] .column.bg-brand-gray[.content[ ```r M1 <- lm(average_price~year*region*type,data=dat_ncr) anova(M1) ``` ``` Analysis of Variance Table Response: average_price Df Sum Sq Mean Sq year 1 18.33 18.33 region 32 305.62 9.55 type 1 754.51 754.51 year:region 32 16.67 0.52 year:type 1 12.66 12.66 region:type 32 62.57 1.96 year:region:type 32 19.35 0.60 Residuals 12374 905.67 0.07 F value Pr(>F) year 250.4643 < 2.2e-16 region 130.4870 < 2.2e-16 type 10308.7792 < 2.2e-16 year:region 7.1172 < 2.2e-16 year:type 172.9561 < 2.2e-16 region:type 26.7147 < 2.2e-16 year:region:type 8.2626 < 2.2e-16 Residuals year *** region *** type *** year:region *** year:type *** region:type *** year:region:type *** Residuals --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ``` ]] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Model diagnostic ]] .row[.split-50[ .column[.content[ ```r autoplot(M1) ``` <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-44-1.svg" style="display: block; margin: auto;" /> ]] .column.bg-brand-gray[.content[ * Model diagnostics are difficult for large data as seen here. * For large sample size, `\(p\)`-values tend to be all significant. * Thus it is more effective to check the effect size instead. .scroll-box-10[ ```r #print(broom::tidy(M1), n=Inf) print(summary(M1)) ``` ``` Call: lm(formula = average_price ~ year * region * type, data = dat_ncr) Residuals: Min 1Q Median 3Q -1.10050 -0.15799 -0.00652 0.14089 Max 1.51631 Coefficients: Estimate (Intercept) -2.482e+02 year 1.238e-01 regionAtlanta 1.683e+02 regionBoise 6.268e+01 regionBoston -9.625e+00 regionCalifornia 9.249e+01 regionCharlotte 6.243e+01 regionChicago -8.891e+01 regionColumbus 1.719e+02 regionDenver 1.372e+02 regionDetroit 1.318e+02 regionGrandRapids -1.167e+02 regionHouston 2.246e+02 regionIndianapolis 2.000e+02 regionJacksonville 5.361e+01 regionLasVegas 1.384e+02 regionLosAngeles 6.812e+01 regionLouisville 1.371e+02 regionNashville 2.341e+02 regionNewYork 3.941e+01 regionOrlando 6.309e+01 regionPhiladelphia 9.511e+01 regionPittsburgh 2.720e+02 regionPortland 1.577e+02 regionRoanoke 1.799e+02 regionSacramento 1.016e+02 regionSanDiego 6.248e+01 regionSanFrancisco 1.537e+02 regionSeattle 7.416e+01 regionSouthCarolina 9.320e+01 regionSpokane 1.501e+02 regionStLouis 1.394e+02 regionSyracuse 1.890e+02 regionTampa 1.432e+01 typeorganic 4.395e+02 year:regionAtlanta -8.360e-02 year:regionBoise -3.122e-02 year:regionBoston 4.752e-03 year:regionCalifornia -4.600e-02 year:regionCharlotte -3.100e-02 year:regionChicago 4.411e-02 year:regionColumbus -8.538e-02 year:regionDenver -6.818e-02 year:regionDetroit -6.551e-02 year:regionGrandRapids 5.789e-02 year:regionHouston -1.116e-01 year:regionIndianapolis -9.928e-02 year:regionJacksonville -2.667e-02 year:regionLasVegas -6.882e-02 year:regionLosAngeles -3.397e-02 year:regionLouisville -6.814e-02 year:regionNashville -1.163e-01 year:regionNewYork -1.952e-02 year:regionOrlando -3.136e-02 year:regionPhiladelphia -4.715e-02 year:regionPittsburgh -1.349e-01 year:regionPortland -7.839e-02 year:regionRoanoke -8.936e-02 year:regionSacramento -5.044e-02 year:regionSanDiego -3.113e-02 year:regionSanFrancisco -7.622e-02 year:regionSeattle -3.687e-02 year:regionSouthCarolina -4.632e-02 year:regionSpokane -7.455e-02 year:regionStLouis -6.922e-02 year:regionSyracuse -9.373e-02 year:regionTampa -7.173e-03 year:typeorganic -2.178e-01 regionAtlanta:typeorganic -3.152e+02 regionBoise:typeorganic -3.302e+02 regionBoston:typeorganic -2.437e+02 regionCalifornia:typeorganic -4.455e+02 regionCharlotte:typeorganic -2.616e+02 regionChicago:typeorganic -2.060e+02 regionColumbus:typeorganic -2.921e+02 regionDenver:typeorganic -4.978e+02 regionDetroit:typeorganic -1.434e+02 regionGrandRapids:typeorganic 1.078e+02 regionHouston:typeorganic -4.664e+02 regionIndianapolis:typeorganic -2.292e+02 regionJacksonville:typeorganic -2.529e+02 regionLasVegas:typeorganic -2.952e+02 regionLosAngeles:typeorganic -4.850e+02 regionLouisville:typeorganic -2.925e+02 regionNashville:typeorganic -3.641e+02 regionNewYork:typeorganic -1.537e+02 regionOrlando:typeorganic -2.266e+02 regionPhiladelphia:typeorganic -2.267e+02 regionPittsburgh:typeorganic -3.644e+02 regionPortland:typeorganic -4.890e+02 regionRoanoke:typeorganic -5.122e+02 regionSacramento:typeorganic -4.997e+02 regionSanDiego:typeorganic -5.577e+02 regionSanFrancisco:typeorganic -5.706e+02 regionSeattle:typeorganic -3.495e+02 regionSouthCarolina:typeorganic -7.648e+01 regionSpokane:typeorganic -5.180e+02 regionStLouis:typeorganic -3.703e+02 regionSyracuse:typeorganic -2.298e+02 regionTampa:typeorganic -2.929e+02 year:regionAtlanta:typeorganic 1.564e-01 year:regionBoise:typeorganic 1.638e-01 year:regionBoston:typeorganic 1.209e-01 year:regionCalifornia:typeorganic 2.210e-01 year:regionCharlotte:typeorganic 1.298e-01 year:regionChicago:typeorganic 1.022e-01 year:regionColumbus:typeorganic 1.449e-01 year:regionDenver:typeorganic 2.468e-01 year:regionDetroit:typeorganic 7.105e-02 year:regionGrandRapids:typeorganic -5.348e-02 year:regionHouston:typeorganic 2.314e-01 year:regionIndianapolis:typeorganic 1.136e-01 year:regionJacksonville:typeorganic 1.256e-01 year:regionLasVegas:typeorganic 1.466e-01 year:regionLosAngeles:typeorganic 2.406e-01 year:regionLouisville:typeorganic 1.450e-01 year:regionNashville:typeorganic 1.806e-01 year:regionNewYork:typeorganic 7.637e-02 year:regionOrlando:typeorganic 1.125e-01 year:regionPhiladelphia:typeorganic 1.125e-01 year:regionPittsburgh:typeorganic 1.806e-01 year:regionPortland:typeorganic 2.426e-01 year:regionRoanoke:typeorganic 2.540e-01 year:regionSacramento:typeorganic 2.480e-01 year:regionSanDiego:typeorganic 2.768e-01 year:regionSanFrancisco:typeorganic 2.832e-01 year:regionSeattle:typeorganic 1.734e-01 year:regionSouthCarolina:typeorganic 3.798e-02 year:regionSpokane:typeorganic 2.570e-01 year:regionStLouis:typeorganic 1.837e-01 year:regionSyracuse:typeorganic 1.139e-01 year:regionTampa:typeorganic 1.453e-01 Std. Error (Intercept) 2.577e+01 year 1.278e-02 regionAtlanta 5.155e+01 regionBoise 5.155e+01 regionBoston 5.155e+01 regionCalifornia 5.155e+01 regionCharlotte 5.155e+01 regionChicago 5.155e+01 regionColumbus 5.155e+01 regionDenver 5.155e+01 regionDetroit 5.155e+01 regionGrandRapids 5.155e+01 regionHouston 5.155e+01 regionIndianapolis 5.155e+01 regionJacksonville 4.075e+01 regionLasVegas 5.155e+01 regionLosAngeles 5.155e+01 regionLouisville 5.155e+01 regionNashville 5.155e+01 regionNewYork 5.155e+01 regionOrlando 5.155e+01 regionPhiladelphia 5.155e+01 regionPittsburgh 5.155e+01 regionPortland 4.075e+01 regionRoanoke 5.155e+01 regionSacramento 5.155e+01 regionSanDiego 5.155e+01 regionSanFrancisco 5.155e+01 regionSeattle 5.155e+01 regionSouthCarolina 5.155e+01 regionSpokane 5.155e+01 regionStLouis 5.155e+01 regionSyracuse 5.155e+01 regionTampa 5.155e+01 typeorganic 3.645e+01 year:regionAtlanta 2.557e-02 year:regionBoise 2.557e-02 year:regionBoston 2.557e-02 year:regionCalifornia 2.557e-02 year:regionCharlotte 2.557e-02 year:regionChicago 2.557e-02 year:regionColumbus 2.557e-02 year:regionDenver 2.557e-02 year:regionDetroit 2.557e-02 year:regionGrandRapids 2.557e-02 year:regionHouston 2.557e-02 year:regionIndianapolis 2.557e-02 year:regionJacksonville 2.021e-02 year:regionLasVegas 2.557e-02 year:regionLosAngeles 2.557e-02 year:regionLouisville 2.557e-02 year:regionNashville 2.557e-02 year:regionNewYork 2.557e-02 year:regionOrlando 2.557e-02 year:regionPhiladelphia 2.557e-02 year:regionPittsburgh 2.557e-02 year:regionPortland 2.021e-02 year:regionRoanoke 2.557e-02 year:regionSacramento 2.557e-02 year:regionSanDiego 2.557e-02 year:regionSanFrancisco 2.557e-02 year:regionSeattle 2.557e-02 year:regionSouthCarolina 2.557e-02 year:regionSpokane 2.557e-02 year:regionStLouis 2.557e-02 year:regionSyracuse 2.557e-02 year:regionTampa 2.557e-02 year:typeorganic 1.808e-02 regionAtlanta:typeorganic 7.290e+01 regionBoise:typeorganic 7.290e+01 regionBoston:typeorganic 7.290e+01 regionCalifornia:typeorganic 7.290e+01 regionCharlotte:typeorganic 7.290e+01 regionChicago:typeorganic 7.290e+01 regionColumbus:typeorganic 7.290e+01 regionDenver:typeorganic 7.290e+01 regionDetroit:typeorganic 7.290e+01 regionGrandRapids:typeorganic 7.290e+01 regionHouston:typeorganic 7.290e+01 regionIndianapolis:typeorganic 7.290e+01 regionJacksonville:typeorganic 5.763e+01 regionLasVegas:typeorganic 7.290e+01 regionLosAngeles:typeorganic 7.290e+01 regionLouisville:typeorganic 7.290e+01 regionNashville:typeorganic 7.290e+01 regionNewYork:typeorganic 7.290e+01 regionOrlando:typeorganic 7.290e+01 regionPhiladelphia:typeorganic 7.290e+01 regionPittsburgh:typeorganic 7.290e+01 regionPortland:typeorganic 5.763e+01 regionRoanoke:typeorganic 7.290e+01 regionSacramento:typeorganic 7.290e+01 regionSanDiego:typeorganic 7.290e+01 regionSanFrancisco:typeorganic 7.290e+01 regionSeattle:typeorganic 7.290e+01 regionSouthCarolina:typeorganic 7.290e+01 regionSpokane:typeorganic 7.290e+01 regionStLouis:typeorganic 7.290e+01 regionSyracuse:typeorganic 7.290e+01 regionTampa:typeorganic 7.290e+01 year:regionAtlanta:typeorganic 3.616e-02 year:regionBoise:typeorganic 3.616e-02 year:regionBoston:typeorganic 3.616e-02 year:regionCalifornia:typeorganic 3.616e-02 year:regionCharlotte:typeorganic 3.616e-02 year:regionChicago:typeorganic 3.616e-02 year:regionColumbus:typeorganic 3.616e-02 year:regionDenver:typeorganic 3.616e-02 year:regionDetroit:typeorganic 3.616e-02 year:regionGrandRapids:typeorganic 3.616e-02 year:regionHouston:typeorganic 3.616e-02 year:regionIndianapolis:typeorganic 3.616e-02 year:regionJacksonville:typeorganic 2.858e-02 year:regionLasVegas:typeorganic 3.616e-02 year:regionLosAngeles:typeorganic 3.616e-02 year:regionLouisville:typeorganic 3.616e-02 year:regionNashville:typeorganic 3.616e-02 year:regionNewYork:typeorganic 3.616e-02 year:regionOrlando:typeorganic 3.616e-02 year:regionPhiladelphia:typeorganic 3.616e-02 year:regionPittsburgh:typeorganic 3.616e-02 year:regionPortland:typeorganic 2.858e-02 year:regionRoanoke:typeorganic 3.616e-02 year:regionSacramento:typeorganic 3.616e-02 year:regionSanDiego:typeorganic 3.616e-02 year:regionSanFrancisco:typeorganic 3.616e-02 year:regionSeattle:typeorganic 3.616e-02 year:regionSouthCarolina:typeorganic 3.616e-02 year:regionSpokane:typeorganic 3.616e-02 year:regionStLouis:typeorganic 3.616e-02 year:regionSyracuse:typeorganic 3.616e-02 year:regionTampa:typeorganic 3.616e-02 t value (Intercept) -9.631 year 9.683 regionAtlanta 3.264 regionBoise 1.216 regionBoston -0.187 regionCalifornia 1.794 regionCharlotte 1.211 regionChicago -1.725 regionColumbus 3.334 regionDenver 2.662 regionDetroit 2.558 regionGrandRapids -2.265 regionHouston 4.357 regionIndianapolis 3.879 regionJacksonville 1.316 regionLasVegas 2.685 regionLosAngeles 1.322 regionLouisville 2.661 regionNashville 4.542 regionNewYork 0.765 regionOrlando 1.224 regionPhiladelphia 1.845 regionPittsburgh 5.276 regionPortland 3.871 regionRoanoke 3.490 regionSacramento 1.971 regionSanDiego 1.212 regionSanFrancisco 2.982 regionSeattle 1.439 regionSouthCarolina 1.808 regionSpokane 2.911 regionStLouis 2.704 regionSyracuse 3.667 regionTampa 0.278 typeorganic 12.058 year:regionAtlanta -3.270 year:regionBoise -1.221 year:regionBoston 0.186 year:regionCalifornia -1.799 year:regionCharlotte -1.213 year:regionChicago 1.725 year:regionColumbus -3.340 year:regionDenver -2.667 year:regionDetroit -2.562 year:regionGrandRapids 2.264 year:regionHouston -4.367 year:regionIndianapolis -3.883 year:regionJacksonville -1.319 year:regionLasVegas -2.692 year:regionLosAngeles -1.329 year:regionLouisville -2.665 year:regionNashville -4.549 year:regionNewYork -0.763 year:regionOrlando -1.227 year:regionPhiladelphia -1.844 year:regionPittsburgh -5.278 year:regionPortland -3.878 year:regionRoanoke -3.495 year:regionSacramento -1.973 year:regionSanDiego -1.218 year:regionSanFrancisco -2.981 year:regionSeattle -1.442 year:regionSouthCarolina -1.812 year:regionSpokane -2.916 year:regionStLouis -2.707 year:regionSyracuse -3.666 year:regionTampa -0.281 year:typeorganic -12.046 regionAtlanta:typeorganic -4.323 regionBoise:typeorganic -4.529 regionBoston:typeorganic -3.343 regionCalifornia:typeorganic -6.111 regionCharlotte:typeorganic -3.588 regionChicago:typeorganic -2.826 regionColumbus:typeorganic -4.007 regionDenver:typeorganic -6.829 regionDetroit:typeorganic -1.967 regionGrandRapids:typeorganic 1.478 regionHouston:typeorganic -6.399 regionIndianapolis:typeorganic -3.144 regionJacksonville:typeorganic -4.389 regionLasVegas:typeorganic -4.050 regionLosAngeles:typeorganic -6.653 regionLouisville:typeorganic -4.013 regionNashville:typeorganic -4.995 regionNewYork:typeorganic -2.109 regionOrlando:typeorganic -3.109 regionPhiladelphia:typeorganic -3.110 regionPittsburgh:typeorganic -4.999 regionPortland:typeorganic -8.486 regionRoanoke:typeorganic -7.026 regionSacramento:typeorganic -6.855 regionSanDiego:typeorganic -7.651 regionSanFrancisco:typeorganic -7.827 regionSeattle:typeorganic -4.795 regionSouthCarolina:typeorganic -1.049 regionSpokane:typeorganic -7.105 regionStLouis:typeorganic -5.080 regionSyracuse:typeorganic -3.152 regionTampa:typeorganic -4.018 year:regionAtlanta:typeorganic 4.325 year:regionBoise:typeorganic 4.531 year:regionBoston:typeorganic 3.343 year:regionCalifornia:typeorganic 6.113 year:regionCharlotte:typeorganic 3.591 year:regionChicago:typeorganic 2.826 year:regionColumbus:typeorganic 4.007 year:regionDenver:typeorganic 6.827 year:regionDetroit:typeorganic 1.965 year:regionGrandRapids:typeorganic -1.479 year:regionHouston:typeorganic 6.399 year:regionIndianapolis:typeorganic 3.143 year:regionJacksonville:typeorganic 4.393 year:regionLasVegas:typeorganic 4.054 year:regionLosAngeles:typeorganic 6.653 year:regionLouisville:typeorganic 4.012 year:regionNashville:typeorganic 4.995 year:regionNewYork:typeorganic 2.112 year:regionOrlando:typeorganic 3.111 year:regionPhiladelphia:typeorganic 3.111 year:regionPittsburgh:typeorganic 4.996 year:regionPortland:typeorganic 8.488 year:regionRoanoke:typeorganic 7.024 year:regionSacramento:typeorganic 6.859 year:regionSanDiego:typeorganic 7.654 year:regionSanFrancisco:typeorganic 7.833 year:regionSeattle:typeorganic 4.797 year:regionSouthCarolina:typeorganic 1.050 year:regionSpokane:typeorganic 7.109 year:regionStLouis:typeorganic 5.081 year:regionSyracuse:typeorganic 3.150 year:regionTampa:typeorganic 4.018 Pr(>|t|) (Intercept) < 2e-16 year < 2e-16 regionAtlanta 0.001100 regionBoise 0.223990 regionBoston 0.851878 regionCalifornia 0.072784 regionCharlotte 0.225885 regionChicago 0.084590 regionColumbus 0.000858 regionDenver 0.007785 regionDetroit 0.010544 regionGrandRapids 0.023542 regionHouston 1.33e-05 regionIndianapolis 0.000105 regionJacksonville 0.188359 regionLasVegas 0.007255 regionLosAngeles 0.186319 regionLouisville 0.007808 regionNashville 5.61e-06 regionNewYork 0.444567 regionOrlando 0.220980 regionPhiladelphia 0.065048 regionPittsburgh 1.34e-07 regionPortland 0.000109 regionRoanoke 0.000484 regionSacramento 0.048711 regionSanDiego 0.225489 regionSanFrancisco 0.002867 regionSeattle 0.150242 regionSouthCarolina 0.070627 regionSpokane 0.003604 regionStLouis 0.006855 regionSyracuse 0.000247 regionTampa 0.781227 typeorganic < 2e-16 year:regionAtlanta 0.001079 year:regionBoise 0.221981 year:regionBoston 0.852553 year:regionCalifornia 0.072034 year:regionCharlotte 0.225337 year:regionChicago 0.084518 year:regionColumbus 0.000841 year:regionDenver 0.007663 year:regionDetroit 0.010413 year:regionGrandRapids 0.023570 year:regionHouston 1.27e-05 year:regionIndianapolis 0.000104 year:regionJacksonville 0.187084 year:regionLasVegas 0.007115 year:regionLosAngeles 0.183924 year:regionLouisville 0.007699 year:regionNashville 5.44e-06 year:regionNewYork 0.445184 year:regionOrlando 0.219999 year:regionPhiladelphia 0.065183 year:regionPittsburgh 1.33e-07 year:regionPortland 0.000106 year:regionRoanoke 0.000475 year:regionSacramento 0.048546 year:regionSanDiego 0.223363 year:regionSanFrancisco 0.002875 year:regionSeattle 0.149260 year:regionSouthCarolina 0.070017 year:regionSpokane 0.003552 year:regionStLouis 0.006790 year:regionSyracuse 0.000247 year:regionTampa 0.779036 year:typeorganic < 2e-16 regionAtlanta:typeorganic 1.55e-05 regionBoise:typeorganic 5.97e-06 regionBoston:typeorganic 0.000831 regionCalifornia:typeorganic 1.02e-09 regionCharlotte:typeorganic 0.000334 regionChicago:typeorganic 0.004718 regionColumbus:typeorganic 6.18e-05 regionDenver:typeorganic 8.96e-12 regionDetroit:typeorganic 0.049246 regionGrandRapids:typeorganic 0.139367 regionHouston:typeorganic 1.63e-10 regionIndianapolis:typeorganic 0.001669 regionJacksonville:typeorganic 1.15e-05 regionLasVegas:typeorganic 5.15e-05 regionLosAngeles:typeorganic 3.00e-11 regionLouisville:typeorganic 6.04e-05 regionNashville:typeorganic 5.95e-07 regionNewYork:typeorganic 0.034961 regionOrlando:typeorganic 0.001881 regionPhiladelphia:typeorganic 0.001873 regionPittsburgh:typeorganic 5.84e-07 regionPortland:typeorganic < 2e-16 regionRoanoke:typeorganic 2.24e-12 regionSacramento:typeorganic 7.48e-12 regionSanDiego:typeorganic 2.14e-14 regionSanFrancisco:typeorganic 5.39e-15 regionSeattle:typeorganic 1.65e-06 regionSouthCarolina:typeorganic 0.294110 regionSpokane:typeorganic 1.27e-12 regionStLouis:typeorganic 3.82e-07 regionSyracuse:typeorganic 0.001625 regionTampa:typeorganic 5.89e-05 year:regionAtlanta:typeorganic 1.54e-05 year:regionBoise:typeorganic 5.93e-06 year:regionBoston:typeorganic 0.000830 year:regionCalifornia:typeorganic 1.01e-09 year:regionCharlotte:typeorganic 0.000330 year:regionChicago:typeorganic 0.004728 year:regionColumbus:typeorganic 6.20e-05 year:regionDenver:typeorganic 9.08e-12 year:regionDetroit:typeorganic 0.049437 year:regionGrandRapids:typeorganic 0.139130 year:regionHouston:typeorganic 1.62e-10 year:regionIndianapolis:typeorganic 0.001676 year:regionJacksonville:typeorganic 1.13e-05 year:regionLasVegas:typeorganic 5.06e-05 year:regionLosAngeles:typeorganic 2.99e-11 year:regionLouisville:typeorganic 6.06e-05 year:regionNashville:typeorganic 5.96e-07 year:regionNewYork:typeorganic 0.034694 year:regionOrlando:typeorganic 0.001867 year:regionPhiladelphia:typeorganic 0.001869 year:regionPittsburgh:typeorganic 5.93e-07 year:regionPortland:typeorganic < 2e-16 year:regionRoanoke:typeorganic 2.26e-12 year:regionSacramento:typeorganic 7.29e-12 year:regionSanDiego:typeorganic 2.09e-14 year:regionSanFrancisco:typeorganic 5.17e-15 year:regionSeattle:typeorganic 1.63e-06 year:regionSouthCarolina:typeorganic 0.293545 year:regionSpokane:typeorganic 1.24e-12 year:regionStLouis:typeorganic 3.80e-07 year:regionSyracuse:typeorganic 0.001637 year:regionTampa:typeorganic 5.90e-05 (Intercept) *** year *** regionAtlanta ** regionBoise regionBoston regionCalifornia . regionCharlotte regionChicago . regionColumbus *** regionDenver ** regionDetroit * regionGrandRapids * regionHouston *** regionIndianapolis *** regionJacksonville regionLasVegas ** regionLosAngeles regionLouisville ** regionNashville *** regionNewYork regionOrlando regionPhiladelphia . regionPittsburgh *** regionPortland *** regionRoanoke *** regionSacramento * regionSanDiego regionSanFrancisco ** regionSeattle regionSouthCarolina . regionSpokane ** regionStLouis ** regionSyracuse *** regionTampa typeorganic *** year:regionAtlanta ** year:regionBoise year:regionBoston year:regionCalifornia . year:regionCharlotte year:regionChicago . year:regionColumbus *** year:regionDenver ** year:regionDetroit * year:regionGrandRapids * year:regionHouston *** year:regionIndianapolis *** year:regionJacksonville year:regionLasVegas ** year:regionLosAngeles year:regionLouisville ** year:regionNashville *** year:regionNewYork year:regionOrlando year:regionPhiladelphia . year:regionPittsburgh *** year:regionPortland *** year:regionRoanoke *** year:regionSacramento * year:regionSanDiego year:regionSanFrancisco ** year:regionSeattle year:regionSouthCarolina . year:regionSpokane ** year:regionStLouis ** year:regionSyracuse *** year:regionTampa year:typeorganic *** regionAtlanta:typeorganic *** regionBoise:typeorganic *** regionBoston:typeorganic *** regionCalifornia:typeorganic *** regionCharlotte:typeorganic *** regionChicago:typeorganic ** regionColumbus:typeorganic *** regionDenver:typeorganic *** regionDetroit:typeorganic * regionGrandRapids:typeorganic regionHouston:typeorganic *** regionIndianapolis:typeorganic ** regionJacksonville:typeorganic *** regionLasVegas:typeorganic *** regionLosAngeles:typeorganic *** regionLouisville:typeorganic *** regionNashville:typeorganic *** regionNewYork:typeorganic * regionOrlando:typeorganic ** regionPhiladelphia:typeorganic ** regionPittsburgh:typeorganic *** regionPortland:typeorganic *** regionRoanoke:typeorganic *** regionSacramento:typeorganic *** regionSanDiego:typeorganic *** regionSanFrancisco:typeorganic *** regionSeattle:typeorganic *** regionSouthCarolina:typeorganic regionSpokane:typeorganic *** regionStLouis:typeorganic *** regionSyracuse:typeorganic ** regionTampa:typeorganic *** year:regionAtlanta:typeorganic *** year:regionBoise:typeorganic *** year:regionBoston:typeorganic *** year:regionCalifornia:typeorganic *** year:regionCharlotte:typeorganic *** year:regionChicago:typeorganic ** year:regionColumbus:typeorganic *** year:regionDenver:typeorganic *** year:regionDetroit:typeorganic * year:regionGrandRapids:typeorganic year:regionHouston:typeorganic *** year:regionIndianapolis:typeorganic ** year:regionJacksonville:typeorganic *** year:regionLasVegas:typeorganic *** year:regionLosAngeles:typeorganic *** year:regionLouisville:typeorganic *** year:regionNashville:typeorganic *** year:regionNewYork:typeorganic * year:regionOrlando:typeorganic ** year:regionPhiladelphia:typeorganic ** year:regionPittsburgh:typeorganic *** year:regionPortland:typeorganic *** year:regionRoanoke:typeorganic *** year:regionSacramento:typeorganic *** year:regionSanDiego:typeorganic *** year:regionSanFrancisco:typeorganic *** year:regionSeattle:typeorganic *** year:regionSouthCarolina:typeorganic year:regionSpokane:typeorganic *** year:regionStLouis:typeorganic *** year:regionSyracuse:typeorganic ** year:regionTampa:typeorganic *** --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual standard error: 0.2705 on 12374 degrees of freedom Multiple R-squared: 0.5678, Adjusted R-squared: 0.5632 F-statistic: 124.1 on 131 and 12374 DF, p-value: < 2.2e-16 ``` ] ]] ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Estimate for Conventional Avocado Price in 2015 July ]] .row[.content[ <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-46-1.svg" style="display: block; margin: auto;" /> ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Estimate for Conventional Avocado Price in 2016 July ]] .row[.content[ <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-47-1.svg" style="display: block; margin: auto;" /> ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Estimate for Conventional Avocado Price in 2017 July ]] .row[.content[ <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-48-1.svg" style="display: block; margin: auto;" /> ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Estimate for Organic Avocado Price in 2015 July ]] .row[.content[ <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-49-1.svg" style="display: block; margin: auto;" /> ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Estimate for Organic Avocado Price in 2016 July ]] .row[.content[ <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-50-1.svg" style="display: block; margin: auto;" /> ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Estimate for Organic Avocado Price in 2017 July ]] .row[.content[ <img src="lecture21_2020JC_files/figure-html/unnamed-chunk-51-1.svg" style="display: block; margin: auto;" /> ]] --- class: split-10 .row.bg-brand-blue.white[.content.vmiddle[ # Wait... ]] .row[.split-50[ .column[.content[ * What about the different PLU, Price Look-Up code? * What about different total volumes etc? * You can try to predict these values for July 2019. * Some models have simplifying assumptions that the volume is constant across year or all regions are increasing their volume by the same amount. ]] .column.bg-brand-gray[.content.vmiddle.center[ {{content}} ]] ]]