This lab corresponds with the course material covering basic text analysis (see associated class material from February 4 and February 11). For each question please be sure to read the directions carefully. The questions will indicate specific techniques and application selections so that you as students can feel fully confident in knowing this topic. Each question should be answered fully including both your code as well as sone type of output to show that your code worked properly (ex. using a head() call to display the results of a query). If code isn’t needed to answer the question, your written output should display the same process as those that do have code + output - in other words, please be sure to justify your answers so that it is clear why you are giving that answer and, more specifically, make sure each part of the question is fully answered by your response.
This lab will count for 10 points total (division of these points is indicated by each portion below) and is due EOD on February 13. Submissions should be received through Dropbox and each student should submit at minimum a HTML or PDF file of their knit R Markdown file (be sure to check that your options are set to output your knit files so that we can see code and output that are required for answers). In order to recieve full credit you must turn in a PDF or HTML of your .RMD file individually by the due date with all your code and cells run. You may work with others but each student is expected to turn in their own lab submissions each week.
In addition to loading any packages needed to answer the questions below, please load the dataset described here to your environment. We will use the Movie Review data frame the text2vec package for this assignment. To load this dataset, run the code below shown below. Note that we will use the “review” column where each value or review is an individual document within our analysis.
library(text2vec)
# confirm upload of data frame
glimpse(movie_review)## Observations: 5,000
## Variables: 3
## $ id <chr> "5814_8", "2381_9", "7759_3", "3630_4", "9495_8", "819…
## $ sentiment <int> 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, …
## $ review <chr> "With all this stuff going down at the moment with MJ …head(movie_review$review)## [1] "With all this stuff going down at the moment with MJ i've started listening to his music, watching the odd documentary here and there, watched The Wiz and watched Moonwalker again. Maybe i just want to get a certain insight into this guy who i thought was really cool in the eighties just to maybe make up my mind whether he is guilty or innocent. Moonwalker is part biography, part feature film which i remember going to see at the cinema when it was originally released. Some of it has subtle messages about MJ's feeling towards the press and also the obvious message of drugs are bad m'kay.<br /><br />Visually impressive but of course this is all about Michael Jackson so unless you remotely like MJ in anyway then you are going to hate this and find it boring. Some may call MJ an egotist for consenting to the making of this movie BUT MJ and most of his fans would say that he made it for the fans which if true is really nice of him.<br /><br />The actual feature film bit when it finally starts is only on for 20 minutes or so excluding the Smooth Criminal sequence and Joe Pesci is convincing as a psychopathic all powerful drug lord. Why he wants MJ dead so bad is beyond me. Because MJ overheard his plans? Nah, Joe Pesci's character ranted that he wanted people to know it is he who is supplying drugs etc so i dunno, maybe he just hates MJ's music.<br /><br />Lots of cool things in this like MJ turning into a car and a robot and the whole Speed Demon sequence. Also, the director must have had the patience of a saint when it came to filming the kiddy Bad sequence as usually directors hate working with one kid let alone a whole bunch of them performing a complex dance scene.<br /><br />Bottom line, this movie is for people who like MJ on one level or another (which i think is most people). If not, then stay away. It does try and give off a wholesome message and ironically MJ's bestest buddy in this movie is a girl! Michael Jackson is truly one of the most talented people ever to grace this planet but is he guilty? Well, with all the attention i've gave this subject....hmmm well i don't know because people can be different behind closed doors, i know this for a fact. He is either an extremely nice but stupid guy or one of the most sickest liars. I hope he is not the latter."
## [2] "\\\"The Classic War of the Worlds\\\" by Timothy Hines is a very entertaining film that obviously goes to great effort and lengths to faithfully recreate H. G. Wells' classic book. Mr. Hines succeeds in doing so. I, and those who watched his film with me, appreciated the fact that it was not the standard, predictable Hollywood fare that comes out every year, e.g. the Spielberg version with Tom Cruise that had only the slightest resemblance to the book. Obviously, everyone looks for different things in a movie. Those who envision themselves as amateur \\\"critics\\\" look only to criticize everything they can. Others rate a movie on more important bases,like being entertained, which is why most people never agree with the \\\"critics\\\". We enjoyed the effort Mr. Hines put into being faithful to H.G. Wells' classic novel, and we found it to be very entertaining. This made it easy to overlook what the \\\"critics\\\" perceive to be its shortcomings."
## [3] "The film starts with a manager (Nicholas Bell) giving welcome investors (Robert Carradine) to Primal Park . A secret project mutating a primal animal using fossilized DNA, like Jurassik Park, and some scientists resurrect one of nature's most fearsome predators, the Sabretooth tiger or Smilodon . Scientific ambition turns deadly, however, and when the high voltage fence is opened the creature escape and begins savagely stalking its prey - the human visitors , tourists and scientific.Meanwhile some youngsters enter in the restricted area of the security center and are attacked by a pack of large pre-historical animals which are deadlier and bigger . In addition , a security agent (Stacy Haiduk) and her mate (Brian Wimmer) fight hardly against the carnivorous Smilodons. The Sabretooths, themselves , of course, are the real star stars and they are astounding terrifyingly though not convincing. The giant animals savagely are stalking its prey and the group run afoul and fight against one nature's most fearsome predators. Furthermore a third Sabretooth more dangerous and slow stalks its victims.<br /><br />The movie delivers the goods with lots of blood and gore as beheading, hair-raising chills,full of scares when the Sabretooths appear with mediocre special effects.The story provides exciting and stirring entertainment but it results to be quite boring .The giant animals are majority made by computer generator and seem totally lousy .Middling performances though the players reacting appropriately to becoming food.Actors give vigorously physical performances dodging the beasts ,running,bound and leaps or dangling over walls . And it packs a ridiculous final deadly scene. No for small kids by realistic,gory and violent attack scenes . Other films about Sabretooths or Smilodon are the following : Sabretooth(2002)by James R Hickox with Vanessa Angel, David Keith and John Rhys Davies and the much better 10.000 BC(2006) by Roland Emmerich with with Steven Strait, Cliff Curtis and Camilla Belle. This motion picture filled with bloody moments is badly directed by George Miller and with no originality because takes too many elements from previous films. Miller is an Australian director usually working for television (Tidal wave, Journey to the center of the earth, and many others) and occasionally for cinema ( The man from Snowy river, Zeus and Roxanne,Robinson Crusoe ). Rating : Below average, bottom of barrel."
## [4] "It must be assumed that those who praised this film (\\\"the greatest filmed opera ever,\\\" didn't I read somewhere?) either don't care for opera, don't care for Wagner, or don't care about anything except their desire to appear Cultured. Either as a representation of Wagner's swan-song, or as a movie, this strikes me as an unmitigated disaster, with a leaden reading of the score matched to a tricksy, lugubrious realisation of the text.<br /><br />It's questionable that people with ideas as to what an opera (or, for that matter, a play, especially one by Shakespeare) is \\\"about\\\" should be allowed anywhere near a theatre or film studio; Syberberg, very fashionably, but without the smallest justification from Wagner's text, decided that Parsifal is \\\"about\\\" bisexual integration, so that the title character, in the latter stages, transmutes into a kind of beatnik babe, though one who continues to sing high tenor -- few if any of the actors in the film are the singers, and we get a double dose of Armin Jordan, the conductor, who is seen as the face (but not heard as the voice) of Amfortas, and also appears monstrously in double exposure as a kind of Batonzilla or Conductor Who Ate Monsalvat during the playing of the Good Friday music -- in which, by the way, the transcendant loveliness of nature is represented by a scattering of shopworn and flaccid crocuses stuck in ill-laid turf, an expedient which baffles me. In the theatre we sometimes have to piece out such imperfections with our thoughts, but I can't think why Syberberg couldn't splice in, for Parsifal and Gurnemanz, mountain pasture as lush as was provided for Julie Andrews in Sound of Music...<br /><br />The sound is hard to endure, the high voices and the trumpets in particular possessing an aural glare that adds another sort of fatigue to our impatience with the uninspired conducting and paralytic unfolding of the ritual. Someone in another review mentioned the 1951 Bayreuth recording, and Knappertsbusch, though his tempi are often very slow, had what Jordan altogether lacks, a sense of pulse, a feeling for the ebb and flow of the music -- and, after half a century, the orchestral sound in that set, in modern pressings, is still superior to this film."
## [5] "Superbly trashy and wondrously unpretentious 80's exploitation, hooray! The pre-credits opening sequences somewhat give the false impression that we're dealing with a serious and harrowing drama, but you need not fear because barely ten minutes later we're up until our necks in nonsensical chainsaw battles, rough fist-fights, lurid dialogs and gratuitous nudity! Bo and Ingrid are two orphaned siblings with an unusually close and even slightly perverted relationship. Can you imagine playfully ripping off the towel that covers your sister's naked body and then stare at her unshaven genitals for several whole minutes? Well, Bo does that to his sister and, judging by her dubbed laughter, she doesn't mind at all. Sick, dude! Anyway, as kids they fled from Russia with their parents, but nasty soldiers brutally slaughtered mommy and daddy. A friendly smuggler took custody over them, however, and even raised and trained Bo and Ingrid into expert smugglers. When the actual plot lifts off, 20 years later, they're facing their ultimate quest as the mythical and incredibly valuable White Fire diamond is coincidentally found in a mine. Very few things in life ever made as little sense as the plot and narrative structure of \\\"White Fire\\\", but it sure is a lot of fun to watch. Most of the time you have no clue who's beating up who or for what cause (and I bet the actors understood even less) but whatever! The violence is magnificently grotesque and every single plot twist is pleasingly retarded. The script goes totally bonkers beyond repair when suddenly and I won't reveal for what reason Bo needs a replacement for Ingrid and Fred Williamson enters the scene with a big cigar in his mouth and his sleazy black fingers all over the local prostitutes. Bo's principal opponent is an Italian chick with big breasts but a hideous accent, the preposterous but catchy theme song plays at least a dozen times throughout the film, there's the obligatory \\\"we're-falling-in-love\\\" montage and loads of other attractions! My God, what a brilliant experience. The original French title translates itself as \\\"Life to Survive\\\", which is uniquely appropriate because it makes just as much sense as the rest of the movie: None!"
## [6] "I dont know why people think this is such a bad movie. Its got a pretty good plot, some good action, and the change of location for Harry does not hurt either. Sure some of its offensive and gratuitous but this is not the only movie like that. Eastwood is in good form as Dirty Harry, and I liked Pat Hingle in this movie as the small town cop. If you liked DIRTY HARRY, then you should see this one, its a lot better than THE DEAD POOL. 4/5"(1 point) Before cleaning or processing data, it is good to have an idea of what things you should look out for in this specific case. Using just the output of “reviews” shown above, describe an example of one of the processing steps described in class clearly shown in our data. (In other words, what issues do you see specifically with this data? Ex. numbers are used in each of the 5 reviews, so they may be important to consider for our analysis.)
(1 point) Create a corpus of this dataframe.
(1 point) Create a new dataframe to store our reviews in tidytext format.
(2 points) Using your corpus, perform the associated pre-processing step: stopwords, punctuation, numbers, word case, white space, stemming. If this step does not apply in this case given the data format we are using, indicate that is the case:
(2 points) Using your tidytext data object, perform the same text pre-processing steps indicated above. If this step does not apply in this case given the data format we are using, indicate that is the case:
(1 point) Create and inspect a document term matrix using your corpus object.
(1 point) Create and inspect a document term matrix using your tidytext object.
(1 point) Describe one major difference between using a corpus or tidytext as it would effect your analysis.