Software Quality Assurance & Testing Asked on October 25, 2021
I am trying to extract the tables generated by selecting "Branches", a city and a district from this site: https://www.acb.com.vn/wps/portal/en/atm
So far, I have been able to write the code to parse through each city and district:
from selenium.webdriver.support.ui import Select
from selenium.webdriver import Chrome
import pandas as pd
import time
webdriver = "chromedriver.exe"
driver = Chrome(webdriver)
driver.get('https://www.acb.com.vn/wps/portal/en/atm')
branch_selector = driver.find_element_by_xpath('//*[@id="branch"]')
branch_selector.click()
city = Select(driver.find_element_by_id('cityId'))
for i in range(len(city.options)):
city.select_by_index(i)
time.sleep(1)
district = Select(driver.find_element_by_id('districtId'))
for j in range(len(district.options)):
district.select_by_index(j)
time.sleep(1)
try:
find_btn = driver.find_element_by_xpath('//*[@id="frm-filter"]/div[3]/a[1]')
find_btn.click()
time.sleep(1)
except:
close_btn = driver.find_element_by_xpath('//*[@id="close-send-email"]/span[2]')
close_btn.click()
time.sleep(1)
Now, I want to extract the table that’s displayed in each iteration of the 2 loops. However, if you look at the HTML for the table, it does not make use of the "table" tag:
So, how do I extract the table for each city-district pair?
I tried the following:
try:
click_btn = driver.find_element_by_xpath('//*[@id="frm-filter"]/div[3]/a[1]')
click_btn.click()
time.sleep(1)
table = driver.find_elements_by_class_name('tbody')
for table_row in table:
row = table_row.find_elements_by_class_name('row')
print ([r.text for r in row])
except:
close_btn = driver.find_element_by_xpath('//*[@id="close-send-email"]/span[2]')
close_btn.click()
time.sleep(1)
But it prints a list of blank elements for each city-district pair, the length of the list being as many addresses are present in the table for the corresponding city-district pair:
['', '', '', '']
['', '', '', '']
['', '', '', '']
['', '', '', '']
['', '', '', '']
['', '', '', '']
['', '', '', '', '']
['', '', '', '', '']
['', '', '', '', '']
['', '', '', '', '']
['', '', '', '', '']
['', '', '', '', '']
['', '', '', '']
['', '']
['', '']
I also tried to access each element in each row of the table individually:
try:
find_btn = driver.find_element_by_xpath('//*[@id="frm-filter"]/div[3]/a[1]')
find_btn.click()
time.sleep(1)
table = driver.find_elements_by_class_name('tbody')
for table_row in table:
row = table_row.find_elements_by_class_name('row')
for element in row:
time.sleep(1)
Type.append(element.find_element_by_class_name('col type'))
Address.append(element.find_element_by_class_name('col address'))
District.append(element.find_element_by_class_name('col district'))
Tel_Fax.append(element.find_element_by_class_name('col tel-fax'))
Hours.append(element.find_element_by_class_name('col hours'))
except:
close_btn = driver.find_element_by_xpath('//*[@id="close-send-email"]/span[2]')
close_btn.click()
time.sleep(1)
But this gives the following error:
---------------------------------------------------------------------------
NoSuchElementException Traceback (most recent call last)
<ipython-input-41-2d73f0dc931c> in <module>
39
---> 40 Type.append(element.find_element_by_class_name('col type'))
41 Address.append(element.find_element_by_class_name('col address'))
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="col type"]"}
Since it says css selector in the error, I tried the following:
element.find_element_by_css_selector('div.col.type').text
This outputs a blank string, ''
.
So, how do I do this?
EDIT: The HTML of the table, for one district-city selection, is:
<div class="tbody">
<div class="row" id="row1">
<div class="col stt">1</div>
<div class="col type">
PGD Hai Bà Trưng</div>
<div class="col address">56-58-60 Hai Bà Trưng, P. Bến Nghé, Quan 1, Ho Chi Minh</div>
<div class="col district">1</div>
<div class="col tel-fax">(028) 6291 3690<br>(028) 6291 3691</div>
<div class="col hours"> 07:00-16:30</div>
<div class="col control"><a href="#" title="Direction" class="btn-direction" onclick="showDialogDirection('56-58-60 Hai Bà Trưng, P. Bến Nghé, Quan 1, Ho Chi Minh', '10.77714,106.704325', 1); return false;">Direction</a></div>
</div>
<div class="row" id="row2">
<div class="col stt">2</div>
<div class="col type">
PGD Đa Kao</div>
<div class="col address">45 Võ Thị Sáu, P. Đa Kao, Quan 1, Ho Chi Minh</div>
<div class="col district">1</div>
<div class="col tel-fax">(028) 6290 5980<br>(028) 6290 5981</div>
<div class="col hours"> 07:30 – 16:30</div>
<div class="col control"><a href="#" title="Direction" class="btn-direction" onclick="showDialogDirection('45 Võ Thị Sáu, P. Đa Kao, Quan 1, Ho Chi Minh', '10.790715,106.69486', 2); return false;">Direction</a></div>
</div>
<div class="row" id="row3">
<div class="col stt">3</div>
<div class="col type">
PGD Nguyễn Công Trứ</div>
<div class="col address">74 - 76 Nguyễn Công Trứ, P. Nguyễn Thái Bình, Quan 1, Ho Chi Minh</div>
<div class="col district">1</div>
<div class="col tel-fax">(028) 3914 4470 <br>(028) 3914 4471</div>
<div class="col hours"> 07:30 – 16:30</div>
<div class="col control"><a href="#" title="Direction" class="btn-direction" onclick="showDialogDirection('74 - 76 Nguyễn Công Trứ, P. Nguyễn Thái Bình, Quan 1, Ho Chi Minh', '10.76972,106.703142', 3); return false;">Direction</a></div>
</div>
<div class="row" id="row4">
<div class="col stt">4</div>
<div class="col type">
PGD Lê Lợi</div>
<div class="col address">72 Lê Lợi, P. Bến Thành, Quận 1, TP.Hồ Chí Minh</div>
<div class="col district">1</div>
<div class="col tel-fax">(028) 3821 4619<br>(028) 3821 4618</div>
<div class="col hours"> 07:00-16:30</div>
<div class="col control"><a href="#" title="Direction" class="btn-direction" onclick="showDialogDirection('72 Lê Lợi, P. Bến Thành, Quận 1, TP.Hồ Chí Minh', '10.773541,106.699635', 4); return false;">Direction</a></div>
</div>
<div class="row" id="row5">
<div class="col stt">5</div>
<div class="col type">
CN Sài Gòn</div>
<div class="col address">41 Mạc Đỉnh Chi, P. Đakao, Quan 1, Ho Chi Minh</div>
<div class="col district">1</div>
<div class="col tel-fax">(028) 3824 3770<br>(028) 3824 3946</div>
<div class="col hours"> 07:30 – 16:30</div>
<div class="col control"><a href="#" title="Direction" class="btn-direction" onclick="showDialogDirection('41 Mạc Đỉnh Chi, P. Đakao, Quan 1, Ho Chi Minh', '10.786191,106.697818', 5); return false;">Direction</a></div>
</div>
<div class="row" id="row6">
<div class="col stt">6</div>
<div class="col type">
PGD Nguyễn Thái Bình</div>
<div class="col address">176 – 178 Ký Con, P. Nguyễn Thái Bình, Quan 1, Ho Chi Minh</div>
<div class="col district">1</div>
<div class="col tel-fax">(028) 3915 1310<br>(028) 3915 1311</div>
<div class="col hours"> 07:30 – 16:30</div>
<div class="col control"><a href="#" title="Direction" class="btn-direction" onclick="showDialogDirection('176 – 178 Ký Con, P. Nguyễn Thái Bình, Quan 1, Ho Chi Minh', '10.768917,106.696863', 6); return false;">Direction</a></div>
</div>
<div class="row" id="row7">
<div class="col stt">7</div>
<div class="col type">
PGD Bến Chương Dương</div>
<div class="col address">328 Võ Văn Kiệt, phường Cô Giang, Quận 1, Tp.HCM</div>
<div class="col district">1</div>
<div class="col tel-fax">(028) 3837 0586<br>(028) 3837 0584</div>
<div class="col hours"> 7h30-16h30</div>
<div class="col control"><a href="#" title="Direction" class="btn-direction" onclick="showDialogDirection('328 Võ Văn Kiệt, phường Cô Giang, Quận 1, Tp.HCM', '10.76161,106.695998', 7); return false;">Direction</a></div>
</div>
<div class="row" id="row8">
<div class="col stt">8</div>
<div class="col type">
PGD Trần Khắc Chân</div>
<div class="col address">48-50 Nguyễn Hữu Cầu, P.Tân Định, Q.1, TP.HCM</div>
<div class="col district">1</div>
<div class="col tel-fax">(028) 3820 9990<br>(028) 3526 7738</div>
<div class="col hours"> 07:30 -16:30</div>
<div class="col control"><a href="#" title="Direction" class="btn-direction" onclick="showDialogDirection('48-50 Nguyễn Hữu Cầu, P.Tân Định, Q.1, TP.HCM', '10.790724, 106.690976', 8); return false;">Direction</a></div>
</div>
<div class="row" id="row9">
<div class="col stt">9</div>
<div class="col type">
PGD Cống Quỳnh</div>
<div class="col address">106 108 Cống Quỳnh, P. Nguyễn Cư Trinh, Q.1</div>
<div class="col district">1</div>
<div class="col tel-fax">(028) 38385464<br>(028) 3925 6645</div>
<div class="col hours"> 07:30 -16:30</div>
<div class="col control"><a href="#" title="Direction" class="btn-direction" onclick="showDialogDirection('106 108 Cống Quỳnh, P. Nguyễn Cư Trinh, Q.1', '10.764772,106.687505', 9); return false;">Direction</a></div>
</div>
<div class="row" id="row10">
<div class="col stt">10</div>
<div class="col type">
CN Bến Thành</div>
<div class="col address">96 Lý Tự Trọng, P. Bến Thành, Quan 1, Ho Chi Minh</div>
<div class="col district">1</div>
<div class="col tel-fax">(028) 3825 7949<br>(028) 3825 7950</div>
<div class="col hours"> 07:30-16:30</div>
<div class="col control"><a href="#" title="Direction" class="btn-direction" onclick="showDialogDirection('96 Lý Tự Trọng, P. Bến Thành, Quan 1, Ho Chi Minh', '10.774379, 106.697395', 10); return false;">Direction</a></div>
</div>
<div class="row" id="row11">
<div class="col stt">11</div>
<div class="col type">
PGD Tân Định </div>
<div class="col address">261 Trần Quang Khải, Phường Tân Định, Quận 1, TP.HCM</div>
<div class="col district">1</div>
<div class="col tel-fax">(028) 3848 0520<br></div>
<div class="col hours"> 07:30 - 16:30</div>
<div class="col control"><a href="#" title="Direction" class="btn-direction" onclick="showDialogDirection('261 Trần Quang Khải, Phường Tân Định, Quận 1, TP.HCM', '10.791284, 106.688080', 11); return false;">Direction</a></div>
</div>
<div class="row" id="row12">
<div class="col stt">12</div>
<div class="col type">
PGD Nguyễn Du</div>
<div class="col address">Tầng hầm 1, tầng trệt, tầng lửng và tầng 2 tòa nhà 480 đường Nguyễn Thị Minh Khai, Phường 2, Quận 3, TP.Hồ Chí Minh</div>
<div class="col district">1</div>
<div class="col tel-fax">(028) 35218626<br>(028) 35218627</div>
<div class="col hours"> 07:30 -16:30</div>
<div class="col control"><a href="#" title="Direction" class="btn-direction" onclick="showDialogDirection('Tầng hầm 1, tầng trệt, tầng lửng và tầng 2 tòa nhà 480 đường Nguyễn Thị Minh Khai, Phường 2, Quận 3, TP.Hồ Chí Minh', '10.777328,106.698459', 12); return false;">Direction</a></div>
</div>
</div>
Your problem might be sames as https://stackoverflow.com/questions/20888592/gettext-method-of-selenium-chrome-driver-sometimes-returns-an-empty-string
But upon initial checking on the attributes of your element, I would suggest to use "outerText" attribute. You can check your element's properties in developer's tool Console by typing $x("your_xpath").
Answered by Millie Anne Volante on October 25, 2021
Get help from others!
Recent Questions
Recent Answers
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP