Skip to content

Commit

Permalink
Supports custom robots tags
Browse files Browse the repository at this point in the history
  • Loading branch information
dlackty committed Jan 27, 2023
1 parent 17fab0f commit 72d79f5
Show file tree
Hide file tree
Showing 5 changed files with 82 additions and 0 deletions.
33 changes: 33 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,9 @@ Use these options to customize the title format:
| `:og` | add Open Graph tags (Hash) |
| `:twitter` | add Twitter tags (Hash) |
| `:refresh` | refresh interval and optionally url to redirect to |
| `:robots` | add custom robots tags (Hash) |
| `:googlebot` | add custom googlebot tags (Hash) |
| `:bingbot` | add custom bingbot tags (Hash) |

And here are a few examples to give you ideas.

Expand Down Expand Up @@ -758,6 +761,36 @@ Further reading:

- [App Links Documentation](https://developers.facebook.com/docs/applinks)

### Robots

Besides using noindex, there's still other custom robots met tags options to instruct search engine to serve crawled site content in specific ways.

```ruby
set_meta_tags robots: {
"max-snippet": -1,
"max-video-preview": -1
}

# <meta name="robots" content="max-snippet:-1, max-video-preview:-1">

set_meta_tags googlebot: {
"unavailable_after": "2020-09-21"
}

# <meta name="googlebot" content="unavailable_after:2020-09-21">

set_meta_tags bingbot: {
"max-image-preview": "large"
}

# <meta name="bingbot" content="max-image-preview:large" />
```

Further reading:

* [Robots meta tag, data-nosnippet, and X-Robots-Tag specifications](https://developers.google.com/search/reference/robots_meta_tag)
* [Robots Metatags](https://www.bing.com/webmasters/help/which-robots-metatags-does-bing-support-5198d240)

### Custom meta tags

Starting from version 1.3.1, you can specify arbitrary meta tags, and they will
Expand Down
7 changes: 7 additions & 0 deletions lib/meta_tags/meta_tags_collection.rb
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,13 @@ def extract_robots
calculate_robots_attributes(result, attributes)
end

[:robots, :googlebot, :bingbot].each do |bot|
values = extract(bot).presence
if values
result[bot.to_s].concat(values.map { |k, v| "#{k}:#{v}" })
end
end

result.transform_values { |v| v.join(", ") }
end

Expand Down
3 changes: 3 additions & 0 deletions lib/meta_tags/view_helper.rb
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,9 @@ def refresh(refresh)
# @option default [String, Integer] :refresh (nil) meta refresh tag;
# @option default [Hash] :open_graph ({}) add Open Graph meta tags.
# @option default [Hash] :open_search ({}) add Open Search link tag.
# @option default [Hash] :robots ({}) add robots meta tags.
# @option default [Hash] :googlebot ({}) add googlebot meta tags.
# @option default [Hash] :bingbot ({}) add bingbot meta tags.
# @return [String] HTML meta tags to render in HEAD section of the
# HTML document.
#
Expand Down
3 changes: 3 additions & 0 deletions sig/lib/meta_tags/view_helper.rbs
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ module MetaTags
# open_search: Hash[Renderer::meta_key, Renderer::meta_value],
# article: Hash[Renderer::meta_key, Renderer::meta_value],
# al: Hash[Renderer::meta_key, Renderer::meta_value],
# robots: Hash[Renderer::meta_key, Renderer::meta_value],
# googlebot: Hash[Renderer::meta_key, Renderer::meta_value],
# bingbot: Hash[Renderer::meta_key, Renderer::meta_value],
# refresh: Integer | String | nil,
# } & Hash[Renderer::meta_key, Renderer::meta_value]
type meta_tags = Hash[Renderer::meta_key, Renderer::meta_value]
Expand Down
36 changes: 36 additions & 0 deletions spec/view_helper/robots_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# frozen_string_literal: true

require "spec_helper"

RSpec.describe MetaTags::ViewHelper, "displaying robots meta tags" do
it "displays meta tags specified with :robots" do
subject.display_meta_tags(robots: {"max-snippet": -1}).tap do |meta|
expect(meta).to have_tag("meta", with: {content: "max-snippet:-1", name: "robots"})
end
end

it "displays meta tags specified with :googlebot" do
subject.display_meta_tags(googlebot: {unavailable_after: "2020-09-21"}).tap do |meta|
expect(meta).to have_tag("meta", with: {content: "unavailable_after:2020-09-21", name: "googlebot"})
end
end

it "displays meta tags specified with :bingbot" do
subject.display_meta_tags(bingbot: {"max-image-preview": "large"}).tap do |meta|
expect(meta).to have_tag("meta", with: {content: "max-image-preview:large", name: "bingbot"})
end
end

it "displays multiple custom robots tags in a hash" do
subject.display_meta_tags(robots: {"max-snippet": -1, "max-video-preview": -1}).tap do |meta|
expect(meta).to have_tag("meta", with: {content: "max-snippet:-1, max-video-preview:-1", name: "robots"})
end
end

it "displays custom robots tags along with noindex" do
subject.noindex(true)
subject.display_meta_tags(robots: {"max-snippet": -1, "max-video-preview": -1}).tap do |meta|
expect(meta).to have_tag("meta", with: {content: "noindex, max-snippet:-1, max-video-preview:-1", name: "robots"})
end
end
end

0 comments on commit 72d79f5

Please sign in to comment.