dev-resources.site
for different kinds of informations.
[Beginner] How to build Youtube video crawler web application with Rails 6, Devise, Nokogiri and Bootstrap 4?
In this tutorial, you will get a Funny Youtube Videos web application with Ruby on Rails up and running on your local server and will push your work into Github. How to make it alive? If you are casually browsing and are maybe interested in doing a Ruby on Rails tutorial, stop what you are doing RIGHT NOW and give this podcast a listen! It will give you the fuel you need to continue down the web developer path; it definitely fueled me.
Reading and executing tutorials is an amazing way to learn, so roll up your sleeves, take a slug of coffee, and let’s dive in! This tutorial assumes you have Ruby on Rails installed and all other pre-requisites, like github and a text editor and such.
I have put my github of tutorial at the end of the article.
1. Design our web application
Here, I have been design our app with 2 main layouts:
- List youtubes videos
- Share youtube video url
- Not signed in page
- Singed in page
- Share youtube video page
2. Building our web application
2.1 Setup Rails
As most Ruby on Rails fans might be aware, Rails 6 is coming soon, and bringing a number of eagerly awaited features and changes. For starters, remember that Rails 6 requires Ruby 2.5+ . So, make sure you have a plan to upgrade your systems accordingly, in case you have not done so already.
We will use Postgres for our project. So let write:
rails new funny-youtube-videos --database=postgresql
A whole bunch of cool stuff happens in the terminal:
And wait amoment until it run out. Then:
cd funny-youtube-videos
rails s
Now, navigate to localhost/3000 to make sure everything is working:
Yay! You’re on Rails!
Next, we will use Devise gem for authentication. If you don't know Devise, please go to https://github.com/plataformatec/devise, and read. This gem is very helpful,it help you save a lot time with how to control authentication for web application.
Add the following line to your Gemfile:
gem 'devise'
Then run bundle install
Next, you need to run the generator:
$ rails generate devise:install
In our application, User is model contains user's information. Next, we will setup devise with User:
rails generate devise User
Then runrails db:migrate
Now, check our folder models and db/schema.rb we got User table with: email, password, etc... Restart application server again.
Next step, we need table to storage youtube video shared by users. So we will create Post model with fields under:
- Id : integer
- Title: string
- Share Url: string
- Description: text
- User Id : who share youtube video
That's Ok! We run rails g model Post title:string share_url:string description:text user:references . We got migrate file look likes:
class CreatePosts < ActiveRecord::Migration[6.0]
def change
create_table :posts do |t|
t.string :share_url, null: false
t.string :title
t.text :description
t.references :user, null: false, foreign_key: true
t.timestamps
end
end
end
2.2 Building controllers, views.
In our application, we need PostsController & UsersController. Open our config/routes.rb and adding bellow:
Rails.application.routes.draw do
root 'posts#index'
resources :posts
devise_for :users
post 'access' => 'users#access', as: :create_session
get 'share' => 'posts#new', as: :share_movie
end
In our UsersController we will control session with Devise. Help anonymous can sign up & sign in our application. At access , when anonymous enter correct their & password match with our users, it will create session and they signed in successfully. Opposite, it will create new user with email & password, then signed in.
def access
if valid_user_params?
user = User.find_for_authentication(email: user_params[:email])
valid_auth = user && user&.valid_password?(user_params[:password])
user = User.create(user_params) unless valid_auth
end
if user
sign_in(user)
flash[:success] = "You have been access successfully!"
else
flash[:error] = "Invalid email or password. Please try again!"
end
redirect_to root_path
end
At PostsController we have
a. index list youtube videos
@posts = Post.order(id: :desc)
We will pagination for our list with gem will_paginate. Add our Gemfile gem 'will_paginate', '~> 3.1.0' then bundle install. Now:
@posts = Post.order(id: :desc).paginate(page: params[:page], per_page: 10)
b. create post when cuser submit their youtube url at share page. I have been write script help we can get youtube video information such as: title, description. I use nokogiri to do it. nokogiri is a Rubygem providing HTML, XML, SAX, and Reader parsers with XPath and CSS selector support. We can use it to crawling websites, ... etc.
I write my script in a service object app/services/post_service.rb
require 'nokogiri'
require 'open-uri'
class PostService
def initialize(current_user, params)
@params = params
@current_user = current_user
end
def create
post = Post.new(@params)
post.user = @current_user
if valid_share_url?
doc = get_doc(@params[:share_url]) if @params[:share_url].present?
post.title = doc.css('title')&.first&.content&.strip
post.description = doc.search('#watch-description-text')&.first&.content
else
post.errors.add(:share_url, 'Invalid Youtube url')
end
post
end
private
def valid_share_url?
@params[:share_url].match(/^(?:https?:\/\/)?(?:www\.)?youtu(?:\.be|be\.com)\/(?:watch\?v=)?([\w-]{10,})/).present?
end
def get_doc(path)
user_agent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.854.0 Safari/535.2"
Nokogiri::HTML.parse(open(path, {'User-Agent' => user_agent}), nil, 'UTF-8')
end
end
So, now our create of PostsController will look like:
@post = PostService.new(current_user, post_params).create
respond_to do |format|
if @post.errors.blank? && @post.save
format.html { redirect_to root_url, notice: 'Your moive was successfully shared. Thank you!' }
format.json { render :show, status: :created, location: @post }
else
flash[:error] = @post.errors.full_messages.first
format.html { render :new }
format.json { render json: @post.errors, status: :unprocessable_entity }
end
end
That's all logic. Now we need implement our web site with front end. We will use Bootstrap for front end. It too easy to use. I have been use slim to write html.
In views/layouts/application.html.slim
doctype html
html
head
meta content=("text/html; charset=UTF-8") http-equiv="Content-Type" /
title Funny Movies Site
= csrf_meta_tags
= csp_meta_tag
= stylesheet_pack_tag 'application', media: 'all', 'data-turbolinks-track': 'reload'
= javascript_pack_tag 'application', 'data-turbolinks-track': 'reload'
body
= render 'shared/nav'
.container
= yield
= render 'shared/footer'
In views/shared/_nav.html.slim
.d-flex.flex-column.flex-md-row.align-items-center.p-3.px-md-4.mb-3.bg-white.border-bottom.box-shadow
.col-sm-5
= link_to root_path do
h3.font-weight-normal Funny Movies Site
.col-sm-7 style="display: flex; justify-content: flex-end;"
- if current_user
button.btn-light Welcome #{current_user.email}
|
= link_to "Share a movie", share_movie_path, class: 'btn btn-success', role: "button"
|
= link_to "Logout", destroy_user_session_path, method: :delete, class: 'btn btn-danger', role: "button"
- else
= form_for :user, method: :post, url: create_session_path, html: { class: 'form-inline' } do |f|
.form-group.mb-2
= f.email_field :email, class: 'form-control', placeholder: 'email'
.form-group.mx-sm-3.mb-2
= f.password_field :password, class: 'form-control', placeholder: 'password'
= f.submit 'Login / Register', class: 'btn btn-primary mb-2'
In home page, also index of PostsController.
views/posts/index.html.slim
.container
- if @posts.empty?
h5 No movies
- @posts.each do |post|
.card.mb-3
.row.no-gutters
.col-md-6
iframe width="420" height="315" src=(post.share_url.gsub('watch?v=', 'embed/'))
.col-md-6
.card-body
h5.card-title #{post.title&.gsub('- YouTube', '')}
p.share Share by: #{post.user.email}
p Description:
p.card-text #{post.description&.truncate(250)}
= will_paginate @posts
In share a youtube page.
views/posts/new.html.slim
.card style=("width: 40rem;margin: auto")
.card-header.text-center
h5 Share a Youtube movie
.card-body
= render 'form'
And views/posts/_form.html.slim
= form_for @post do |f|
.form-group.row
.col-sm-2
= f.label :share_url, class: 'col-form-label'
.col-sm-10
= f.text_field :share_url, class: 'form-control', required: true
.form-group.row
.col-sm-2
.col-sm-10
= f.submit 'Submit', class: 'btn btn-primary btn-lg btn-block'
2.3 Make it alive!
I recommend guys using Heroku for hostingRails application. Heroku is a platform as a service (PaaS) that enables developers to build, run, and operate applications entirely in the cloud. Register now!
This artice will help you deploy our website to Heroku.
Our website alive now: https://funny-movies-site.herokuapp.com
We have awesome funny youtube videos with Rails 6.
Conclusion
In this tutorial, we can build Sharing Youtube Videos with Rails 6 and gems: Devise, nokogiri, will_paginate and Bootstrap 4. Learn how about crawling website. And deploy our website to heroku. Make it alive!
Featured ones: