Skip to content
Toggle navigation
P
Projects
G
Groups
S
Snippets
Help
lichenggang
/
update_cate
This project
Loading...
Sign in
Toggle navigation
Go to a project
Project
Repository
Issues
0
Merge Requests
0
Pipelines
Wiki
Snippets
Settings
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Commit
c37739a2
authored
May 08, 2019
by
lichenggang
Browse files
Options
_('Browse Files')
Download
Email Patches
Plain Diff
add testmart
parent
2919eafc
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
3 deletions
plat/testmart/cate_stock.py
plat/testmart/cate_stock.py
View file @
c37739a2
from
utils.base
import
Module_Base
from
utils.proxy_util
import
proxies
domain
=
'http
:
'
domain
=
'http
s://www.testmart.cn
'
headers
=
{
"User-Agent"
:
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36"
,
}
...
...
@@ -23,7 +23,7 @@ def get_levels(level1_doms):
for
level1_dom
in
level1_doms
:
level1
=
dict
()
level1
[
'cat_name'
]
=
level1_dom
[
0
]
.
xpath
(
'.//a[1]/text()'
)[
0
]
.
strip
()
level1
[
'url'
]
=
level1_dom
[
0
]
.
xpath
(
'.//a[1]/@href'
)[
0
]
level1
[
'url'
]
=
domain
+
level1_dom
[
0
]
.
xpath
(
'.//a[1]/@href'
)[
0
]
level1
[
'level'
]
=
1
level1
[
'islast'
]
=
0
level1
[
'parent_id'
]
=
0
...
...
@@ -32,7 +32,7 @@ def get_levels(level1_doms):
for
level2_dom
in
level1_dom
[
1
]
.
xpath
(
'.//a'
):
level2
=
{}
level2
[
'cat_name'
]
=
level2_dom
.
xpath
(
'./text()'
)[
0
]
.
strip
()
level2
[
'url'
]
=
level2_dom
.
xpath
(
'./@href'
)[
0
]
level2
[
'url'
]
=
domain
+
level2_dom
.
xpath
(
'./@href'
)[
0
]
level2
[
'level'
]
=
2
level2
[
'islast'
]
=
1
level1
[
'level2_list'
]
.
append
(
level2
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment