What is the Vim8 package feature and how should I use it?

The answer provided by @statox is very descriptive, but for a new user that can be distracting, because they could read the help file directly. I want to outline what you need to do in pointers.

1. Create pack/*/start directory under any of the directory provided by set packpath. I did in ~/.config/nvim/pack/*/start. Note that you can use any directory name you want in place of * but you can't omit it totally, I don't know why. For example, you can use directory ~/.config/nvim/pack/foo/start or ~/.config/nvim/pack/bar/start but not ~/.config/nvim/pack/start.

2. Go to the pack/*/start directory and clone the package there.

3. Fire up n/vim and do :scriptnames to check if everything is loaded. Don't worry if not every part is loaded, because some files are meant to loaded after some hook, e.g. autoload/plugin.vim.
4. To uninstall, just remove the directory where you cloned the package. Don't need to do anything else.
5. Do :helptags ALL to generate tags for all the help docs. Do :helptags {dir} for generating tags for the help docs under directory dir. For example, if you put your plugin in ~/.config/nvim/pack/foo/plugin_name, then do :helptags ~/.config/nvim/pack/foo/plugin_name/doc. That will generate a tags file in the doc directory of the plugin. If you remove the plugin from the directory, the tags file will be gone and vim will not find the help file, so you don't need to uninstall the doc file manually.

The new format can be thought of as a Pathogen equivalent so there's still space for a manager that can download the plugins you want. There's a few new plugin managers who leverage this new pack format but I still made Vire since they leave the headache of managing the vimrc to you. If you have multiple machines and want the same config across, Vire makes it super easy.

To expand on the "can it replace plugin managers",

I used to use Vundle, which was fantastic, but now have replaced it with 18 or so lines of bash.

I find it useful to use subfolders in the pack directory to group related plugins. E.g. "Syntax" or "Ruby".

The relevant bash example is below. Place in a file and run it.

Additional discussion around the topic at: https://stories.abletech.nz/get-rid-of-your-vim-plugin-manager-7c8ff742f643#.abnjauzgk

#!/usr/bin/env bash
# This file lives in ~/.vim/pack/install.sh
# Remember to add executable: chmod +x ~/.vim/pack/install.sh
#
# Create new folder in ~/.vim/pack that contains a start folder and cd into it.
#
# Arguments:
#   package_group, a string folder name to create and change into.
#
# Examples:
#   set_group syntax-highlighting
#
function set_group () {
  package_group=$1
  path="$HOME/.vim/pack/$package_group/start"
  mkdir -p "$path"
  cd "$path" || exit
}
# Clone or update a git repo in the current directory.
#
# Arguments:
#   repo_url, a URL to the git repo.
#
# Examples:
#   package https://github.com/tpope/vim-endwise.git
#
function package () {
  repo_url=$1
  expected_repo=$(basename "$repo_url" .git)
  if [ -d "$expected_repo" ]; then
    cd "$expected_repo" || exit
    result=$(git pull --force)
    echo "$expected_repo: $result"
  else
    echo "$expected_repo: Installing..."
    git clone -q "$repo_url"
  fi
}
(
set_group ruby
package https://github.com/tpope/vim-rails.git &
package https://github.com/tpope/vim-rake.git &
package https://github.com/tpope/vim-bundler.git &
package https://github.com/tpope/vim-endwise.git &
wait
) &
(
set_group syntax
package https://github.com/kchmck/vim-coffee-script.git &
package https://github.com/tpope/vim-markdown.git &
package https://github.com/ap/vim-css-color.git &
wait
) &
(
set_group colorschemes
package https://github.com/altercation/vim-colors-solarized.git &
wait
) &
wait

First of all, the relevant documentation can be found with :h packages on the newly compiled Vim8 version and here on Github.

A first important note is about the vocabulary: In Vim8 a package is defined like this:

A Vim package is a directory that contains one or more plugins.

This means that the new package manager was created to help users manage all of their plugins in the same archive. The doc lists the following advantages:

• A package can be downloaded as an archive and unpacked in its own directory. Thus the files are not mixed with files of other plugins. That makes it easy to update and remove.

• A package can be a git, mercurial, etc. repository. That makes it really easy to update.

• A package can contain multiple plugins that depend on each other.

• A package can contain plugins that are automatically loaded on startup and ones that are only loaded when needed with :packadd.

So the idea is to create a folder containing all of the plugins with the following structure:

$HOME/.vim/pack/my-plugins/
                        start/
                            foo/
                                plugin/
                                    foo.vim
                                syntax/
                                    some.vim
                            bar/
                                plugin/
                                    bar.vim
                        opt/
                            buzz/
                                plugin/
                                    buzz.vim

The emplacement of the folder is defined by the option packpath (See :h 'packpath').

Note the importance of the structure of your folder:

• The start folder contains plugins which will be loaded automatically on startup.
• The opt folder contains "optional" plugins, loaded with the packadd command.
• The subfolders (plugin, autoload, doc,...) are the ones you're used to in the plugins.

Here is a recap of the folders:

start/foobar/plugin/foo.vim     " always loaded, defines commands
start/foobar/plugin/bar.vim     " always loaded, defines commands
start/foobar/autoload/foo.vim   " loaded when foo command used
start/foobar/doc/foo.txt        " help for foo.vim
start/foobar/doc/tags           " help tags
opt/fooextra/plugin/extra.vim   " optional plugin, defines commands
opt/fooextra/autoload/extra.vim " loaded when extra command used
opt/fooextra/doc/extra.txt      " help for extra.vim
opt/fooextra/doc/tags           " help tags

Once these files are in the right place, opening Vim will load the plugins in start and make the ones in opt available with :packadd.

---

Now, can this feature replace the existing plugin managers?

Disclaimer: This part may be a little bit opinionated.

I think the approach of this new package manager is really different than the one of the plugins managers we were used to because it is made to manage one (or several) archive(s) containing some plugins.

Out of the box, the package manager doesn't provide features to update your plugins one-by-one, to fetch them automatically from a Github address or to select the plugins you want to enable/disable.

I'm not sure it will be really convenient to use it out of the box (especially because handling nested version control repositories can be a painful task) but maybe this is the occasion to make plugin managers more efficient?

Now it is also possible to imagine moving the existing plugins to adopt the structure required by the package manager and to manage them directly from the file system. Maybe some wrapper will be created to use this new feature.

---

EDIT As suggested by @Sato Katsura here is a note about the helptags command. The Vim8 commit introduced two line in the helptag doc:

:helpt[ags] [++t] {dir}

Generate the help tags file(s) for directory {dir}.
When {dir} is ALL then all "doc" directories in 'runtimepath' will be used.

Which means that the new package manager ease the generation of the helptags put in the user archive. With the single command :helptags ALL all the helptags are generated.