Skip to content
Toggle navigation
P
Projects
G
Groups
S
Snippets
Help
岳巧源
/
jd_data_process
This project
Loading...
Sign in
Toggle navigation
Go to a project
Project
Repository
Issues
0
Merge Requests
0
Pipelines
Wiki
Snippets
Settings
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Commit
7566e7fa
authored
Oct 08, 2024
by
岳巧源
Browse files
Options
_('Browse Files')
Download
Email Patches
Plain Diff
add readme
parent
18059b29
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
12 additions
and
13 deletions
README.md
push_sku_to_nplm.py
README.md
View file @
7566e7fa
主要处理京东分类,属性,品牌等数据
需要定时将京东数据更新到本地db中。
推送组装好的数据到rabbitmq队列中,go_nplm_server 负责消费数据,并将sku发布到京东平台
sku组装过程:
整体架构
1、首先从redis(235主机)中获取到 platform_id, shop_id, supplier_id, brand_id 等信息。
\
2、根据获取到的supplier_id 去db中筛选数据(包含spu_id)
\
3、根据上面筛选到的数据 去redis中 查询对应spu_id 的数据信息。
go_nplm_server中写一个go命令(作为rabbit_mq的 producer)
用于定时获取redis中数据。包含supplier_id, brand_id 等信息。
根据这些信息去mysql中查询符合条件的 goods_id 和 spu_id
上述步骤拿到的goods_id 以及 spu_id 会传送到 rabbit_mq 队列当中。
TODO 暂时未解决(或未确认)的问题:
1.
型号是否是spu_name ??
2.
图片使用基石的默认图片 ??
3.
价格
rabbit_mq的 consumer
根据拿到的
push_sku_to_nplm.py
View file @
7566e7fa
...
...
@@ -15,8 +15,8 @@ def get_channel_info():
"""return supplier_id 或者 brand_id 等等"""
data
=
{}
infos
=
db
.
hgetall
(
'shop_distribution'
)
for
i
in
range
(
len
(
infos
))
:
info_value
=
json
.
loads
(
infos
[
str
(
i
+
1
)
])
for
key
in
infos
:
info_value
=
json
.
loads
(
infos
[
key
])
# 暂时只推京东数据
if
info_value
[
"shop_id"
]
!=
1
or
info_value
[
"platform"
]
!=
1
:
continue
...
...
@@ -50,5 +50,6 @@ def get_class_from_rds(spu_id):
if
__name__
==
'__main__'
:
condition
=
get_channel_info
()
spu_ids
=
period_get_data
(
"liexin_sku_0"
,
"lie_sku_0"
,
condition
)
print
(
spu_ids
)
infos
=
get_class_from_rds
(
spu_ids
[
0
])
print
(
type
(
infos
))
\ No newline at end of file
print
(
infos
)
\ No newline at end of file
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment