PGP, or Pretty Good Privacy, is a software for encryption. It is used to encrypt or sign documents, mails or even disks.
PGP uses the standard OpenPGP.

How it works?

PGP can works with symmetrical or asymmetrical keys. In our case I will explain asymmetrical keys.
An asymmetrical key is a private key, and a public key. The private key is the main key. Like her name, she's private and must not be given to anyone. The private key permit to generate a public key. The public key, is public and can be given to anyone. We can't regenerate the private key from the public key. The keys have a creation date, an expiration date (or not), the mail of the owner, and an ID (like a signature).
The private key can encrypt and decrypt. The public key can only encrypt.
A lot of software allow to create a private and public key.

alt

OK, and what I do with those keys?

In practice, if Bob want to send an email, or encrypted file to Alice, he will need to ask Alice's public key. With this, Bob encrypt the data for Alice and, send it. Only the possessor of the private key, associated to this public key, can decrypt the data, so only Alice will be able to decrypt with his private key. We can't decrypt a data with the public key.

alt

Great. But, how to get the public key?

Yes indeed. If you need to contact Alice, to get the key, it's not very convenient. But PGP servers exists. A PGP server is a catalog of public keys. Everyone can upload his own public key on it. PGP servers are synchronized between them. On the PGP server, you can search for words, and download the public key.
For information: no public key can be removed from a PGP server. If the owner want to disable the published public key, he need to generate a revocation certificate, from his private key, and send it to the PGP server. The public key will continue to appear, but a message will show she's revoked. For example, you can check https://pgp.mit.edu/.

Nice this PGP server... but who can assure me, Bob's public key on the PGP server is really owned by my Bob?

You can't. That's why you need to confirm with the real bob, like face to face, the signature.

OK. I understand how it works. But why can I need to encrypt data?

You're right. Maybe most of the people will don't need to use this, and I'm pretty sure a big part of them didn't even know PGP exists.
You could use encrypted data to send an email with privacy information. Same, you could encrypt secret documents.
In the reality, the networks your emails will travel may not be always encrypted by the connection, and you can't never be sure they are. Using an encrypted mail allow you to do not worry about the quality of the security of the network your emails will use.
Same, you can encrypt it to be sure only Bob can read it.

In a other articles, I will provide some real example of PGP encryption, like how to use encrypted mails on software (like Thunderbird on a PC, or K9-Mail on an Android device), encrypt some data on Gnome, encrypt a disk/partition/container...

(Credits for pictures: Wikipedia / https://en.wikipedia.org/wiki/Public-key_cryptography)