Hello everyone! Today we will tackle a question that sounds like: “Python – how to connect a proxy?”.
Why you might need a proxy is up to you. For example, you may want to scrape data from a website that doesn’t appreciate multiple connections. Or for something else entirely. In any case, you need to connect a proxy. Let’s go!
The code presented below will look as follows:
Establish a connection to a website that shows your current IP.
Output the received data to verify everything is working correctly.
Connect a proxy and establish a connection through it.
Connect to a website that shows your current IP (now considering the proxy :)) and output the received data.
To check our IP address, we will use the site 2ip.ru (which is quite good). Additionally, we’ll use fake browser headers to avoid unnecessary detection.
import requests # Importing the library for handling and creating requests
from bs4 import BeautifulSoup # Importing the library for convenient data searching on websites
headers = ({'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/69.0'}) # Created a fake browser header ;)
Now we have: the required libraries connected and a fake browser header. Next, we’ll initialize a connection to the site to get our current location and IP:
get_location_link = 'https://2ip.ru' # Variable holding the link to the website that determines our IP and location
response = requests.get(url=get_location_link, headers=headers) # Variable storing the website's response (to which we sent our fake browser header as part of the request)
soup = BeautifulSoup(response.text, 'lxml') # Variable containing the parsed website response text
ip = soup.find('div', class_='ip').text.strip() # Variable storing our IP address provided by 2ip.ru (found in a div with the class "ip")
location = soup.find('div', class_='value-country').text.strip() # Variable storing the location (shown on 2ip.ru in a div with the class "value-country")
print(ip, ':', location) # Outputting our IP address and location in the terminal
So, we retrieve and output our current location. Now let’s establish a connection through a proxy server and display the results below for comparison:
print('Now through a proxy:') # Outputting text to the terminal to indicate when the connection is through the proxy
proxy_link = {'https': '8.217.12.240:59394'} # Dictionary specifying the protocol, proxy address, and connection port. Where to find proxies? There are many options, but that's for later.
response = requests.get(url=get_location_link, headers=headers, proxies=proxy_link) # Initializing a connection to the site, but now specifying the proxy in the connection parameters
soup = BeautifulSoup(response.text, 'lxml') # Variable containing the parsed website response text
ip = soup.find('div', class_='ip').text.strip() # Variable storing our IP address provided by 2ip.ru (found in a div with the class "ip")
location = soup.find('div', class_='value-country').text.strip() # Variable storing the location (shown on 2ip.ru in a div with the class "value-country")
print(ip, ':', location) # Outputting our IP address and location in the terminal
If everything was done correctly, you’ll see something like this in the terminal:
The complete code looks like this:
import requests
from bs4 import BeautifulSoup
headers = ({'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/69.0'})
get_location_link = 'https://2ip.ru'
proxy_link = {'https': '8.217.12.240:59394'}
response = requests.get(url=get_location_link, headers=headers)
soup = BeautifulSoup(response.text, 'lxml')
ip = soup.find('div', class_='ip').text.strip()
location = soup.find('div', class_='value-country').text.strip()
print(ip, ':', location)
print('Now through a proxy:')
response = requests.get(url=get_location_link, headers=headers, proxies=proxy_link)
soup = BeautifulSoup(response.text, 'lxml')
ip = soup.find('div', class_='ip').text.strip()
location = soup.find('div', class_='value-country').text.strip()
print(ip, ':', location)
That’s all I can say about “how to connect a proxy”. 🙂 Thanks for your attention! As always, if you have any questions, feel free to write to email or Telegram.
Search queries:
proxy server connection for scraping Python.
Support the Blog!
Running a blog takes a lot of effort, time, and passion. Your donations help improve the content, inspire new ideas, and keep the project going.
If you’ve enjoyed the blog’s materials, any support would mean the world to me. Thank you for being here! ❤️
Donate via PayPal
Donate via Revolut
